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Chapter 1. Linux history 


This chapter briefly tells the history of Unix and where Linux fits in. 


If you are eager to start working with Linux without this blah, blah, blah over history, 
distributions, and licensing then jump straight to Part II - Chapter 8. Working with 
Directories page 73. 


Linux history 


1.1. 1969 


All modern operating systems have their roots in 1969 when Dennis Ritchie and Ken 
Thompson developed the C language and the Unix operating system at AT&T Bell Labs. 
They shared their source code (yes, there was open source back in the Seventies) with the 
rest of the world, including the hippies in Berkeley California. By 1975, when AT&T started 
selling Unix commercially, about half of the source code was written by others. The hippies 
were not happy that a commercial company sold software that they had written; the resulting 
(legal) battle ended in there being two versions of Unix: the official AT&T Unix, and the 
free BSD Unix. 


Development of BSD descendants like FreeBSD, OpenBSD, NetBSD, DragonFly BSD and 
PC-BSD is still active today. 


https://en.wikipedia.org/wiki/Dennis Ritchie 
https://en.wikipedia.org/wiki/Ken Thompson 
https://en.wikipedia.org/wiki/BSD 
https://en.wikipedia.org/wiki/Comparison of BSD operating systems 


1.2. 1980s 


In the Eighties many companies started developing their own Unix: IBM created AIX, Sun 
SunOS (later Solaris), HP HP-UX and about a dozen other companies did the same. The 
result was a mess of Unix dialects and a dozen different ways to do the same thing. And 
here is the first real root of Linux, when Richard Stallman aimed to end this era of Unix 
separation and everybody re-inventing the wheel by starting the GNU project (GNU is Not 
Unix). His goal was to make an operating system that was freely available to everyone, and 
where everyone could work together (like in the Seventies). Many of the command line tools 
that you use today on Linux are GNU tools. 


https://en.wikipedia.org/wiki/Richard Stallman 
https://en.wikipedia.org/wiki/IBM AIX 
https://en.wikipedia.org/wiki/HP-UX 


1.3. 1990s 


The Nineties started with Linus Torvalds, a Swedish speaking Finnish student, buying a 
386 computer and writing a brand new POSIX compliant kernel. He put the source code 
online, thinking it would never support anything but 386 hardware. Many people embraced 
the combination of this kernel with the GNU tools, and the rest, as they say, is history. 


http://en.wikipedia.org/wiki/Linus, Torvalds 
https://en.wikipedia.org/wiki/History of Linux 
https://en.wikipedia.org/wiki/Linux 

https://lwn.net 

http://www.levenez.com/unix/ (a huge Unix history poster) 


Linux history 


1.4. 2015 


Today more than 97 percent of the world's supercomputers (including the complete top 10), 
more than 80 percent of all smartphones, many millions of desktop computers, around 70 
percent of all web servers, a large chunk of tablet computers, and several appliances (dvd- 
players, washing machines, dsl modems, routers, self-driving cars, space station laptops...) 
run Linux. Linux is by far the most commonly used operating system in the world. 


Linux kernel version 4.0 was released in April 2015. Its source code grew by several hundred 
thousand lines (compared to version 3.19 from February 2015) thanks to contributions of 
thousands of developers paid by hundreds of commercial companies including Red Hat, 
Intel, Samsung, Broadcom, Texas Instruments, IBM, Novell, Qualcomm, Nokia, Oracle, 
Google, AMD and even Microsoft (and many more). 
http://kernelnewbies.org/DevelopmentStatistics 


http://kernel.org 
http://www.top500.org 


Chapter 2. distributions 


This chapter gives a short overview of current Linux distributions. 


A Linux distribution is a collection of (usually open source) software on top of a Linux 
kernel. A distribution (or short, distro) can bundle server software, system management 
tools, documentation and many desktop applications in a central secure software 
repository. A distro aims to provide a common look and feel, secure and easy software 
management and often a specific operational purpose. 


Let's take a look at some popular distributions. 


distributions 


2.1. Red Hat 


Red Hat is a billion dollar commercial Linux company that puts a lot of effort in developing 
Linux. They have hundreds of Linux specialists and are known for their excellent support. 
They give their products (Red Hat Enterprise Linux and Fedora) away for free. While Red 
Hat Enterprise Linux (RHEL) is well tested before release and supported for up to seven 
years after release, Fedora is a distro with faster updates but without support. 


2.2. Ubuntu 


Canonical started sending out free compact discs with Ubuntu Linux in 2004 and quickly 
became popular for home users (many switching from Microsoft Windows). Canonical 
wants Ubuntu to be an easy to use graphical Linux desktop without need to ever see a 
command line. Of course they also want to make a profit by selling support for Ubuntu. 


2.3. Debian 


There is no company behind Debian. Instead there are thousands of well organised 
developers that elect a Debian Project Leader every two years. Debian is seen as one of 
the most stable Linux distributions. It is also the basis of every release of Ubuntu. Debian 
comes in three versions: stable, testing and unstable. Every Debian release is named after 
a character in the movie Toy Story. 


2.4. Other 


Distributions like CentOS, Oracle Enterprise Linux and Scientific Linux are based on 
Red Hat Enterprise Linux and share many of the same principles, directories and 
system administration techniques. Linux Mint, Edubuntu and many other *buntu named 
distributions are based on Ubuntu and thus share a lot with Debian. There are hundreds of 
other Linux distributions. 


distributions 


2.5. Which to choose ? 


Below are some very personal opinions on some of the most popular Linux Distributions. 
Keep in mind that any of the below Linux distributions can be a stable server and a nice 
graphical desktop client. 


Table 2.1. choosing a Linux distro 


| distribution name reason(s) for using 
Red Hat Enterprise (RHEL) | You are a manager and you want a good support contract. 
CentOS You want Red Hat without the support contract from Red Hat. 
Fedora You want Red Hat on your laptop/desktop. 
Linux Mint You want a personal graphical desktop to play movies, music and games. 
Debian My personal favorite for servers, laptops, and any other device. 
Ubuntu Very popular, based on Debian, not my favorite. 
Kali You want a pointy-clicky hacking interface. 
| others Advanced users may prefer Arch, Gentoo, OpenSUSE, Scientific, ... 


When you are new to Linux in 2015, go for the latest Mint or Fedora. If you only want to 
practice the Linux command line then install one Debian server and/or one CentOS server 
(without graphical interface). 


Here are some links to help you choose: 


distrowatch.com 
redhat.com 
centos.org 
debian.org 
www.linuxmint.com 
ubuntu.com 


Chapter 3. licensing 


This chapter briefly explains the different licenses used for distributing operating systems 
software. 


Many thanks go to Ywein Van den Brande for writing most of this chapter. 


Y wein is an attorney at law, co-author of The International FOSS Law Book and author 
of Praktijkboek Informaticarecht (in Dutch). 


Ywain Van don Brando. 


Het Praktijkboek Informaticarecht: 
Recht rendeert voor uw onderneming 


over Informaticarecht 
gobundeld in een 
praktische en 
 deskundigo gids. 

The International Free 

and 
Open Source Software 
Law Book Crealaw 


http://ifosslawbook.org 
http://www.crealaw.eu 


licensing 


3.1. about software licenses 


There are two predominant software paradigms: Free and Open Source Software (FOSS) 
and proprietary software. The criteria for differentiation between these two approaches is 
based on control over the software. With proprietary software, control tends to lie more 
with the vendor, while with Free and Open Source Software it tends to be more weighted 
towards the end user. But even though the paradigms differ, they use the same copyright 
laws to reach and enforce their goals. From a legal perspective, Free and Open Source 
Software can be considered as software to which users generally receive more rights via 
their license agreement than they would have with a proprietary software license, yet the 
underlying license mechanisms are the same. 


Legal theory states that the author of FOSS, contrary to the author of public domain 
software, has in no way whatsoever given up his rights on his work. FOSS supports on the 
rights of the author (the copyright) to impose FOSS license conditions. The FOSS license 
conditions need to be respected by the user in the same way as proprietary license conditions. 
Always check your license carefully before you use third party software. 


Examples of proprietary software are AIX from IBM, HP-UX from HP and Oracle 
Database 11g. You are not authorised to install or use this software without paying a 
licensing fee. You are not authorised to distribute copies and you are not authorised to modify 
the closed source code. 


3.2. public domain software and freeware 


Software that is original in the sense that it is an intellectual creation of the author benefits 
copyright protection. Non-original software does not come into consideration for copyright 
protection and can, in principle, be used freely. 


Public domain software is considered as software to which the author has given up all rights 
and on which nobody is able to enforce any rights. This software can be used, reproduced or 
executed freely, without permission or the payment of a fee. Public domain software can in 
certain cases even be presented by third parties as own work, and by modifying the original 
work, third parties can take certain versions of the public domain software out of the public 
domain again. 


Freeware is not public domain software or FOSS. It is proprietary software that you can use 
without paying a license cost. However, the often strict license terms need to be respected. 


Examples of freeware are Adobe Reader, Skype and Command and Conquer: Tiberian 
Sun (this game was sold as proprietary in 1999 and is since 2011 available as freeware). 


3.3. Free Software or Open Source Software 


Both the Free Software (translates to vrije software in Dutch and to Logiciel Libre in 
French) and the Open Source Software movement largely pursue similar goals and endorse 
similar software licenses. But historically, there has been some perception of differentiation 
due to different emphases. Where the Free Software movement focuses on the rights (the 
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four freedoms) which Free Software provides to its users, the Open Source Software 
movement points to its Open Source Definition and the advantages of peer-to-peer software 
development. 


Recently, the term free and open source software or FOSS has arisen as a neutral alternative. 
A lesser-used variant is free/libre/open source software (FLOSS), which uses libre to clarify 
the meaning of free as in freedom rather than as in at no charge. 


Examples of free software are gcc, MySQL and gimp. 


Detailed information about the four freedoms can be found here: 


http://www.gnu.org/philosophy/free-sw.html 


The open source definition can be found at: 


http://www.opensource.org/docs/osd 


The above definition is based on the Debian Free Software Guidelines available here: 


http://www.debian.org/social_contract#guidelines 


3.4. GNU General Public License 


More and more software is being released under the GNU GPL (in 2006 Java was released 
under the GPL). This license (v2 and v3) is the main license endorsed by the Free Software 
Foundation. It’s main characteristic is the copyleft principle. This means that everyone in the 
chain of consecutive users, in return for the right of use that is assigned, needs to distribute 
the improvements he makes to the software and his derivative works under the same 
conditions to other users, if he chooses to distribute such improvements or derivative works. 
In other words, software which incorporates GNU GPL software, needs to be distributed 
in turn as GNU GPL software (or compatible, see below). It is not possible to incorporate 
copyright protected parts of GNU GPL software in a proprietary licensed work. The GPL 
has been upheld in court. 


3.5. using GPLv3 software 


You can use GPLv3 software almost without any conditions. If you solely run the software 
you even don’t have to accept the terms of the GPLv3. However, any other use - such as 
modifying or distributing the software - implies acceptance. 


In case you use the software internally (including over a network), you may modify the 
software without being obliged to distribute your modification. You may hire third parties 
to work on the software exclusively for you and under your direction and control. But if you 
modify the software and use it otherwise than merely internally, this will be considered as 
distribution. You must distribute your modifications under GPLv3 (the copyleft principle). 
Several more obligations apply if you distribute GPLv3 software. Check the GPLv3 license 
carefully. 


You create output with GPLv3 software: The GPLv3 does not automatically apply to the 
output. 


licensing 


3.6. BSD license 


There are several versions of the original Berkeley Distribution License. The most common 
one is the 3-clause license ("New BSD License" or "Modified BSD License"). 


This is a permissive free software license. The license places minimal restrictions on how 
the software can be redistributed. This is in contrast to copyleft licenses such as the GPLv. 
3 discussed above, which have a copyleft mechanism. 


This difference is of less importance when you merely use the software, but kicks in when 
you start redistributing verbatim copies of the software or your own modified versions. 


3.7. other licenses 


FOSS or not, there are many kind of licenses on software. You should read and understand 
them before using any software. 


3.8. combination of software licenses 


When you use several sources or wishes to redistribute your software under a different 
license, you need to verify whether all licenses are compatible. Some FOSS licenses (such 
as BSD) are compatible with proprietary licenses, but most are not. If you detect a license 
incompatibility, you must contact the author to negotiate different license conditions or 
refrain from using the incompatible software. 
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Chapter 4. installing Debian 8 


This module is a step by step demonstration of an actual installation of Debian 8 (also known 
as Jessie). 


We start by downloading an image from the internet and install Debian 8 as a virtual machine 
in Virtualbox. We will also do some basic configuration of this new machine like setting 
an ip address and fixing a hostname. 


This procedure should be very similar for other versions of Debian, and also for distributions 
like Linux Mint, xubuntu/ubuntu/kubuntu or Mepis. This procedure can also be helpful 
if you are using another virtualization solution. 


Go to the next chapter if you want to install CentOS, Fedora, Red Hat Enterprise 
Linux, .... 


installing Debian 8 


4.1. Debian 


Debian is one of the oldest Linux distributions. I use Debian myself on almost every 
computer that I own (including raspbian on the Raspberry Pi). 


Debian comes in releases named after characters in the movie Toy Story. The Jessie release 
contains about 36000 packages. 


Table 4.1. Debian releases 


name number year 
Woody 3.0 2002 
Sarge 3.1 2005 
Etch 4.0 2007 
Lenny 5.0 2009 
Squeeze 6.0 2011 
Wheezy 7 2013 
Jessie 8 2015 


There is never a fixed date for the next Debian release. The next version is released when 
it is ready. 


4.2. Downloading 


All these screenshots were made in November 2014, which means Debian 8 was still in 
‘testing’ (but in 'freeze', so there will be no major changes when it is released). 


Download Debian here: 


Debian -- Getting Debian - Iceweasel ^-nXx 
ile Edit View History Bookmarks Tools Help 


Fi 
e @ https://www.debian.org/di: v @| (Qv DuckDuckGo aixyataQ-= 


(9 Debian -- Getting Debian x | oF | 
Search 


(O About Debian Getting Debian Support Developers' Corner 
debian / getting debian 


Getting Debian 


Debian is distributed freely over Internet. You can download all of it from any of our 
mirrors. The Installation Manual contains detailed installation instructions. 


If you simply want to install Debian, these are your options: 


Download an installation Try Debian live before 
image installing 

Depending on your Internet connection, You can try Debian by booting a live 
vau mav download either nf the svstem from a CD DVD ar LSR kev 
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After a couple of clicks on that website, I ended up downloading Debian 8 (testing) here. It 
should be only one click once Debian 8 is released (somewhere in 2015). 


debian-testing-amd64-lxde-CD-1.iso 2014-11-03 07:35 640M 
2] debian-testing-amd64-netinst.iso 2014-11-03 07:34 231M 


e Index of /cdimage/weekly-builds/amd64/iso-cd - Iceweasel Vom [el os 
File Edit View History Bookmarks Tools Help 

é debian.org/cdimage/weekly-builds/amd64/iso-cd/ ¥ Œ (3v DuckDuckGo (à *86 4 ©- = 
A Index of /cdimage/wee... x | dp | 

LEJ SHA512SUMS 2014-11-03 08:24 14K [4] 
2 SHA512SUMS . sign 2014-11-03 08:24 836 

[2] debian-testing-amd64-CD-1.iso 2014-11-03 08:24 616M 

[2] debian-testing-amd64-CD-2.iso 2014-11-03 08:24 642M 

A debian-testing-amd64-CD-3.iso 2014-11-03 08:24 644M 

A debian-testing-amd64-CD-4.iso 2014-11-03 08:24 647M 

A debian-testing-amd64-CD-5.iso 2014-11-03 08:24 555M 

A debian-testing-amd64-CD-6.iso 2014-11-03 08:24 605M 

Ei debian-testing-amd64-CD-7.iso 2014-11-03 08:24 625M 

A debian-testing-amd64-CD-8.iso 2014-11-03 08:24 644M 

i debian-testing-amd64-kde-CD-1.iso 2014-11-03 07:35 610M 

i 


£J debian-testing-amd64-xfce-CD-1.iso 2014-11-03 07:35 641M 


Apache/2.4.9 (Unix) Server at cdimage.debian.org Port 80 


You have many other options to download and install Debian. We will discuss them much 
later. 


This small screenshot shows the downloading of a netinst .iso file. Most of the software will 
be downloaded during the installation. This also means that you will have the most recent 
version of all packages when the install is finished. 


Library 
4 Ed Organize y Clear Download Search Download: [à 
> O History debian-testing-amd64-netinst.iso 
J Downloads — 


Tags 3 minutes, 32 seconds remaining — 15.5 of 231 MB (1.0 MB/sec) 
> E All Bookmarks 


I already have Debian 8 installed on my laptop (hence the paul ? debian8 prompt). Anyway, 
this is the downloaded file just before starting the installation. 


paul@debian8:~$ ls -hl debian-testing-amd64-netinst.iso 
-rw-r--r-- 1 paul paul 231M Nov 10 17:59 debian-testing-amd64-netinst.iso 
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Create a new virtualbox machine (I already have five, you might have zero for now). Click 
the New button to start a wizard that will help you create a virtual machine. 


W Oracle VM VirtualBox Manager 2. - EX 
File Machine Help 


C e a Y (3 Details (8) Snapshots 


New Settings Start Discard 


(ES) debian7 [El General [El Preview = 
Q @ Powered Off 
Name: RHEL7 
PR winxp Operating System: Red Hat (64 bit) 
E @ Powered Off 
System 
EX" RHEL7 - Base Memory 544 MB RHEL7 
| j O Powered O Processors: 2 
Execution Cap: 90% 
(52) centos7 Boot Order: Hard Disk, CD/DVD 
Q @ Powered Off Acceleration: VT-X/AMDV, Nested Paging, 
PAE/NX 
[4939 server33 
@ Powered Off Display 
Video Memory: 32 MB 
Remote Desktop Server: Disabled 
Video Capture: Disabled 


Storage 


Controller: IDE 

IDE Secondary Master: [CD/DVD] rhel-server-7.0-x«86 64-dvd.iso (3.49 GB) 
Controller: SATA 

SATA Port 0: RHEL7.vdi (Normal, 40.00 GB) 


Qa Audio bd 


The machine needs a name, this screenshot shows that I named it server42. 


Create Virtual Machine 
Name and operating system 


Please choose a descriptive name for the new 
virtual machine and select the type of operating 
system you intend to install on it. The name you 
choose will be used throughout VirtualBox to 
identify this machine. 


Name: [server42] 
Type: [Linux y] (a 
Version: [Debian (64 bit) y] 


Hide Description | < Back | [tex > | Cancel | 
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Most of the defaults in Virtualbox are ok. 
512MB of RAM is enough to practice all the topics in this book. 


Create Virtual Machine 


Memory size 


Select the amount of memory (RAM) in megabytes 
to be allocated to the virtual machine. 


The recommended memory size is 512 MB. 


¿[512 = MB 


A REESE enn 7 


4 MB 8192 MB 


< Back INTIN Cancel | 


We do not care about the virtual disk format. 


Create Virtual Hard Drive 


Hard drive file type 


Please choose the type of file that you would like to use for 
the new virtual hard drive. If you do not need to use it with 
other virtualization software you can leave this setting 
unchanged. 


© VMDK (Virtual Machine Disk) 
C VHD (Virtual Hard Disk) 

© HDD (Parallels Hard Disk) 

C QED (QEMU enhanced disk) 
C QCOW (QEMU Copy-On-Write) 


Hide Description | < Back | [tex > | Cancel | 
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Choosing dynamically allocated will save you some disk space (for a small performance 
hit). 


Create Virtual Hard Drive 
Storage on physical hard drive 


Please choose whether the new virtual hard drive file should 
grow as it is used (dynamically allocated) or if it should be 
created at its maximum size (fixed size). 


A dynamically allocated hard drive file will only use space on 
your physical hard drive as it fills up (up to a maximum fixed 
size), although it will not shrink again automatically when 
space on it is freed. 


A fixed size hard drive file may take longer to create on some 
systems but is often faster to use. 


C Fixed size 


Create Virtual Hard Drive 
File location and size 


Please type the name of the new virtual hard drive file into the 
box below or click on the folder icon to select a different folder 
to create the file in. 


[servera2] a 


Select the size of the virtual hard drive in megabytes. This size 
is the limit on the amount of file data that a virtual machine will 
be able to store on the hard drive. 


J—————— F-89988 


4.00 MB 2.00 TB 


< Back | [_ create | Cancel | 


This finishes the wizard. You virtual machine is almost ready to begin the installation. 
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First, make sure that you attach the downloaded .iso image to the virtual CD drive. (by 
opening Settings, Storage followed by a mouse click on the round CD icon) 


+ server42 - Settings tox 


Storage 


Storage Tee ————————— | Attributes 


[El General 
System 


Display 
€. Controller: IDE er Name: | IDE 

DP Audio © debian-testing-amd64-... Type: [Pixa al 
¿2 Controller: SATA IV Use Host I/O Cache 


@ Network 


= : 
$ Serial Ports [E] server42.vdi 


Shared Folders 


BES 


cancel | Help | 


Personally I also disable sound and usb, because I never use these features. I also remove 
the floppy disk and use a PS/2 mouse pointer. This is probably not very important, but I like 
the idea that it saves some resources. 


Now boot the virtual machine and begin the actual installation. After a couple of seconds 
you should see a screen similar to this. Choose Install to begin the installation of Debian. 


te server42 [Running] - Oracle VM VirtualBox ^-nx 


Machine View Devices Help 


Debian GNU/Linux installer boot menu 
Install 


Graphical iis 
Advanced options » e ld n 


Help 4 
Install with speech o sunthesis GNU/Linux 


Press ENTER to boot or TAB to edit a menu entry 


8dO GE Om () | (9 (€) Left winkey 
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First select the language you want to use. 


te server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


[!!] Select a language 


Choose the language to be used for the installation process. The selected language will 
also be the default language for the installed system. 


Language: 


[E No localization 
Albanian Shqip 
Arabic wot 
Asturian Asturianu 
Basque Euskara 
Belarusian Benapyckaa 
Bosnian Bosanski 
Bulgarian BorrapcKu 
Catalan Catala 
Chinese (Simplified) rh sc (fal te) 
Chinese (Traditional) HX (BEES) 
Croatian Hrvatski 
Czech Čeština 
Danish Dansk 
Dutch Nederlands 


Esperanto Esperanto 
Estonian Eesti 
Finnish Suomi 
French Français 
Galician Galego 
German Deutsch 
Greek EAANU LKü 


<Go Back» 


«Tab» moves; selects; «Enter» activates buttons 


8d O Y Uw (2) [ C9 (8) Left winkey 7 


Choose your country. This information will be used to suggest a download mirror. 


te server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


[!!] Select your location 


The selected location will be used to set your time zone and also for example to help 
select the system locale. Normally this should be the country where you live. 


This is a shortlist of locations based on the language you selected. Choose "other" if 
your location is not listed. 


Country, territory or area: 


Antigua and Barbuda 
Australia 
Botswana 
Canada 

Hong Kong 
India 

Ireland 

New Zealand 
Nigeria 
Philippines 
Singapore 
South Africa 
United Kingdom 
United States 
Zambia 
Zimbabwe 

other 


<Go Back> 


> «Enter: 


BOF Uwe (2) | C9 (8) Left Winkey 7 
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Choose the correct keyboard. On servers this is of no importance since most servers are 
remotely managed via ssh. 


server42 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


Keymap to use: 


Albanian 


E 
LI 
Arabic 
Asturian 
Bangladesh 
Belarusian 
Bengali 
Belgian 
Bosnian 
Brazilian 
+ 


British English 
Bulgarian 

Bulgarian (phonetic layout) 
Canadian French 
Canadian Multilingual 
Catalan 

Chinese 

Croatian 

Czech 

Danish 

Dutch 

Dvorak 

Dzongkha 

Esperanto 

Estonian 

Ethiopian 


<Go Back> 


8d O Y Um (2) [ C9 (€) Left Winkey 7 


server42 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


Please enter the hostname for this system. 

The hostname is a single word that identifies your system to the network. If you don't 
know what your hostname should be, consult your network administrator. If you are setting 
up your own home network, you can make something up here. 

Hostname: 


>. Paull. local 


<Go Back> <Cont inue> 


BO FO mi Y | C9 (2) Left winkey 4 
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Give the root user a password. Remember this password (or use hunter2). 


fe server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


[!!] Set up users and passwords 
You need to set a password for 'root', the system administrative account. A malicious or 
unqualified user with root access can have disastrous results, so you should take care to 
choose a root password that is not easy to guess. It should not be a word found in 
dictionaries, or a word that could be easily associated with you. 


A good password will contain a mixture of letters, numbers and punctuation and should be 
changed at regular intervals. 


The root user should not have an empty password. If you leave this empty, the root 
account will be disabled and the system's initial user account will be given the power to 
become root using the "sudo" command. 

Note that you will not be able to see the password as you type it. 

Root password: 


<Go Back> <Cont inue> 


; «Enter» activates buttons 


8d O Y Umm (2) [ C9 (€) Left winkey 7 


Itis adviced to also create a normal user account. I don't give my full name, Debian 8 accepts 
an identical username and full name paul. 


fe server42 [Running] - Oracle VM VirtualBox m 


Machine View Devices Help 


[!!] Set up users and passwords 


A user account will be created for you to use instead of the root account for 
non-administrative activities. 


Please enter the real name of this user. This information will be used for instance as 
default origin for emails sent by this user as well as any program which displays or uses 
the user's real name. Your full name is a reasonable choice. 


Full name for the new user: 


«Bo Back» «Dont inue» 


activates buttons 


8d O Y ww (2) [ C9 (8) Left Winkey 7 
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The use entire disk refers to the virtual disk that you created before in Virtualbox.. 


fe server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


[!!] Partition disks 


The installer can guide uou through partitioning a disk (using different standard 
schemes) or, if you prefer, you can do it manually. With guided partitioning you will 
still have a chance later to review and customise the results. 


If you choose guided partitioning for an entire disk, you will next be asked which disk 
should be used. 


Partitioning method: 
Guided - u entire disk 
Guided - use entire disk and set up LVM 
Guided - use entire disk and set up encrypted LVM 
Manual 


<Go Back> 


activates buttons 


8 O Y Umm (2) [ C9 (8) Left winkey 7 


Again the default is probably what you want. Only change partitioning if you really know 
what you are doing. 


fe server42 [Running] - Oracle VM VirtualBox m 


Machine View Devices Help 


[!] Partition disks 
Selected for partitioning: 
SCSI3 (0,0,0) (sda) - ATA VBOX HARDDISK: 8.6 GB 


The disk can be partitioned using one of several different schemes. If you are unsure, 
choose the first one. 


Partitioning scheme: 


All files in one partition (recommended for new users) 
Separate /home partition 
Separate /home, ‘var, and /tmp partitions 


«G0 Back» 


Enter» activates buttons 


8d O Y ime (3) [ C9 (8) Left Winkey 7 
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Accept the partition layout (again only change if you really know what you are doing). 


fe server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


[!!] Partition disks 


This is an overview of your currently configured partitions and mount points. Select a 
partition to modify its settings (file system, mount point, etc.), a free space to create 
partitions, or a device to initialize its partition table. 


Guided partitioning 

Configure software RAID 

Configure the Logical Volume Manager 
Configure encrypted volumes 
Configure iSCSI volumes 


SCSI3 (0,0,0) (sda) - 8.6 GB ATA VBOX HARDDISK 
41 primary 8.2 GB f ext4 / 
45 logical 401.6 MB f swap suap 


Undo changes to partitions 
h partitioning and write cha 


«Go Back» 


«Fi» for help; «Tab» moves > lects; <Enter> activates buttons 
BOF Om Y [ C9 (€) Left winkey 7 


This is the point of no return, the magical moment where pressing yes will forever erase 
data on the (virtual) computer. 


fe server42 [Running] - Oracle VM VirtualBox m 


Machine View Devices Help 


[!!] Partition disks 


If you continue, the changes listed below will be written to the disks. Otherwise, you 
will be able to make further changes manually. 


The partition tables of the following devices are changed: 
SCSI3 (0,0,0) (sda) 


The following partitions are going to be formatted: 
partition #1 of SCSI3 (0,0,0) (sda) as ext4 
partition #5 of SCSI3 (0,0,0) (sda) as swap 


Write the changes to disks? 


Enter> activates buttons 


8d O Y Uwe Q | C9 (8) Left winkey 7 
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Software is downloaded from a mirror repository, preferably choose one that is close by (as 
in the same country). 


server42 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


[!] Configure the package manager 


The goal is to find a mirror of the Debian archive that is close to you on the network -- 
be aware that nearby countries, or even your oun, may not be the best choice. 


Debian archive mirror country: 


enter information manually 
Algeria 

Argentina 

Australia 

Austria 

Bangladesh 


Brazil 
Bulgaria 
Canada 
Chile 

China 
Colombia 
Costa Rica 
Croatia 
Czech Republic 
Denmark 

El Salvador 
Estonia 
Finland 
France 


«Go Back» 


ctivates buttons 
8 O Y Umm (2) [ C9 (€) Left Winkey 7 


server42 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


[!] Configure the package manager 


Please select a Debian archive mirror. You should use a mirror in your country or region 
if you do not know which mirror has the best Internet connection to you. 


Usually, ftp.<your country code».debian.org is a good choice. 
Debian archive mirror: 


.belnet.be 
ftp.be.debian.org 
mirror.as35701.net 
http.debian.net 
cdn.debian.net 
ftp.debian.skunet.be 
be.mirror.eurid.eu 


<Go Back> 


; «Enter» activates buttons 


8d O Y Uwe (2) [ C9 (8) Left Winkey 7 
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Leave the proxy field empty (unless you are sure that you are behind a proxy server). 


fe server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


If you need to use a HTTP proxy to access the outside world, enter the proxy information 
here. Otherwise, leave this blank. 


The proxy information should be given in the standard form of 
"http: //[ [user] [:pass] @] host [:port] /". 


HTTP proxy information (blank for none): 


<Go Back> <Cont inue> 


<Tab> mov ects; < "> activates buttons 


BOF Om Y [ C9 (€) Left winkey 7 


Choose whether you want to send anonymous statistics to the Debian project (it gathers data 
about installed packages). You can view the statistics here http://popcon.debian.org/. 


te server42 [Running] - Oracle VM VirtualBox m 


Machine View Devices Help 


The system may anonymously supply the distribution developers with statistics about the 
most used packages on this system. This information influences decisions such as which 
packages should go on the first distribution CD. 


If you choose to participate, the automatic submission script will run once every week, 
sending statistics to the distribution developers. The collected statistics can be vieued 
on http://popcon.debian.onrg/. 


This choice can be later modified by running "dpkg-reconfigure popularity-contest". 


Participate in the package usage survey? 


«Bo Back» 


«Tab» mov 28 ts; «Enter» activates buttons 


8d O Y Uwe (3) | C9 (8) Left Winkey 7 
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Choose what software to install, we do not need any graphical stuff for this training. 


fe server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


At the moment, only the core of the system is installed. To tune the system to your 
needs, you can choose to install one or more of the following predefined collections of 
software. 


Choose software to install: 


Debian desktop environment 
. GNOME 
e 


. Cinnamon 
« MATE 
s LXDE 


system utilities 


«Go Back» 


«Tab» mov ‘ ects; <Enter> activates buttons 


BOF Om Y [ C9 (€) Left winkey 7 


The latest versions are being downloaded. 


te server42 [Running] - Oracle VM VirtualBox ^ 


Machine View Devices Help 


Retrieving file 264 of 265 


BO Bo ma Y [ C9 (€) Left winkey 7 
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Say yes to install the bootloader on the virtual machine. 


ía server42 [Running] - Oracle VM VirtualBox ®t- OX 
Machine View Devices Help 


| [!] Install the GRUB boot loader on a hard disk [ 


It seems that this new installation is the only operating system on this computer. If so, 
it should be safe to install the GRUB boot loader to the master boot record of your first 
hard drive. 


Warning: If the installer failed to detect another operating system that is present on 
your computer, modifying the master boot record will make that operating system 
temporarily unbootable, though GRUB can be manually configured later to boot it. 


Install the GRUB boot loader to the master boot record? 
<Go Back> «No» 


oF O tal (9) | C9 (e) Left Winkey y 


Booting for the first time shows the grub screen 


fe server42 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


GNU GRUB 


which entry is h 


to edit 


oP [1m Y | Co [e] Left Winkey 
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A couple seconds later you should see a lot of text scrolling of the screen (dmesg). After 
which you are presented with this getty and are allowed your first logon. 


ía server42 [Running] - Oracle VM VirtualBox ^-DnX 


Machine View Devices Help 


Debian GNU/Linux jessie/sid s 


server42 login: 


GP (i umi Y | C9 (€) Left winkey 


You should now be able to log on to your virtual machine with the root account. Do you 
remember the password ? Was it hunter2 ? 


ía server42 [Running] - Oracle VM VirtualBox ^-DX 
Machine View Devices Help 


Debian GNU/Linux jessie/sid st 


-amd64 #1 SM Debian 3.16.5 2014-10-10) 


s included with the 
t distribution terms fa escribed in the 
dual files in /usr/share/doc 


comes with ABSOLUTELY NO WARRANTY, to the extent 
permitted by applicable law. 
root@seryver4d2:"# _ 


J) P d e Y | C9 [9] Left winkey 


The screenshots in this book will look like this from now on. You can just type those 
commands in the terminal (after you logged on). 


root@server42:~# who am i 


root ttyl ALA ilo) AL a ik 
root@server42:~# hostname 
server42 


root@server42:~# date 
Mon Nov 10 Silt CET 2014 
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4.3. virtualbox networking 


You can also log on from remote (or from your Windows/Mac/Linux host computer) using 
ssh or putty. Change the network settings in the virtual machine to bridge. This will enable 
your virtual machine to receive an ip address from your local dhcp server. 


The default virtualbox networking is to attach virtual network cards to nat. This screenshiot 
shows the ip address 10.0.2.15 when on nat: 


root@server42:~# ifconfig 

etho Link encap:Ethernet HWaddr 08:00:27:f5:74:cf 

inet adar: 10 0.2.15 Beast: 10702.2551 MASIA oo 25.5110 
inet6 addr: fe80::a00:27ff:fef5:74cf/64 Scope:Link 

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 

RX packets:11 errors:0 dropped:0 overruns:0 frame:0 

TX packets:19 errors:0 dropped:0 overruns:0 carrier:0 
collisions:0 txqueuelen:1000 

RX bytes:2352 (2.2 KiB) TX bytes:1988 (1.9 KiB) 


lo Link encap:Local Loopback 

inet addr: 127-00. Masks255 0900 

inet6 addr: ::1/128 Scope:Host 

UP LOOPBACK RUNNING MTU:65536 Metric:1 

RX packets:0 errors:0 dropped:0 overruns:0 frame:0 
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 
collisions:0 txqueuelen:0 

RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 


server42 - Settings 


General Network 
System 
(E) Display Adapter 1 | Adapter 2 | Adapter 3 | Adapter 4 | 
Bg Storage Jv Enable Network Adapter 
a Audio Attached to: [NAT +] 
Ela Not attached 
Mame; NAT 
Y Advanced 


Adapter Type; 


x] Shared Folders 


Promiscuous Mode: Generic Driver 


MAC Address; [080027F574CF 5) 


Iv Cable Connected 


Port Forwarding | 


cancel | Help | 


By shutting down the network interface and enabling it again, we force Debian to renew an 
ip address from the bridged network. 


root@server42:~# # do not run ifdown while connected over ssh! 
root@server42:~# ifdown ethO 

Killed old client process 

Internet Systems Consortium DHCP Client 4.3.1 

Copyright 2004-2014 Internet Systems Consortium. 

All rights reserved. 

For info, please visit https://www.isc.org/software/dhcp/ 


Listening on LPF/eth0/08:00:27:£5:74:cf 
Sending on LPR eth0/08700:27:f5: 4 et 
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Sending on Socket/fallback 

DHCPRELEASE on eth0 to 10.0.2.2 port 67 

root@server42:~# # now enable bridge in virtualbox settings 
root@server42:~# ifup ethO 

Internet Systems Consortium DHCP Client 4.3.1 

Copyright 2004-2014 Internet Systems Consortium. 

All rights reserved. 

For info, please visit https://www.isc.org/software/dhcp/ 


Listening on LPF/eth0/08:00:27:£5:74:cf 

Sending on LPF/eth0/08:00:27:£5:74:cf 

Sending on Socket/fallback 

DECPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8 
DECPDISCOVER on ethO to 255.255.255.255 port 67 interval 8 
DHCPREQUEST on echo to 255.255.255.255 port 67 

DHCPOEFER from 192.168.11.42 

DHCEACK Crom 192m6 TAZ 


bound to 192.168.1.111 -- renewal in 2938 seconds. 
root@server42:~# ifconfig ethO 
etho Link encap:Ethernet HWaddr 08:00:27:f5:74:cf 


Inet addr: 192 os EcasiT92 U6 Zoom Masia 2557255102 55100 
inet6 addr: fe80::a00:27ff:fef5:74cf/64 Scope:Link 

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 

RX packets:15 errors:0 dropped:0 overruns:0 frame:0 

TX packets:31 errors:0 dropped:0 overruns:0 carrier:0 
collisions:0 txqueuelen:1000 

RX bytes:3156 (3.0 KiB) TX bytes:3722 (3.6 KiB) 
root@server42:~# 


Here is an example of ssh to this freshly installed computer. Note that Debian 8 has disabled 
remote root access, so i need to use the normal user account. 


paul@debian8:~$ ssh paul@192.168.1.111 
paul@192.168.1.111's password: 


The programs included with the Debian GNU/Linux system are free software; 
the exact distribution terms for each program are described in the 
individual files in /usr/share/doc/*/copyright. 


Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent 
permitted by applicable law. 

paul8server42:-$ 

paul@server42:~$ su - 

Password: 

root@server42:~# 


TODO: putty screenshot here... 
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4.4. setting the hostname 


The hostname of the server is asked during installation, so there is no need to configure this 
manually. 


root@server42:~# hostname 

server42 

root@server42:~# cat /etc/hostname 

server42 

root@server42:~# dnsdomainname 

paul.local 

root@server42:~# grep server42 /etc/hosts 

O ell server42.paul.local server42 
root@server42:~# 


4.5. adding a static ip address 


This example shows how to add a static ip address to your server. 


You can use ifconfig to set a static address that is active until the next reboot (or until the 
next ifdown). 
a 


root@server42:~# ifconfig eth0:0 10.104.33.39 


Adding a couple of lines to the /etc/network/interfaces file to enable an extra ip address 
forever. 


root@server42:~# vi /etc/network/interfaces 

root@server42:~# tail -4 /etc/network/interfaces 

auto eth0:0 

iface eth0:0 inet static 

address 10.104.33.39 

netmask 255.255.0.0 

root@server42:~# ifconfig 

etho Link encap:Ethernet  HWaddr 08:00:27:f5:74:cf 

inet addr: T92 65" il, BcastE: 9» Too 255 Maske 255925/5 259590 
inet6 addr: fe80::a00:27ff:fef5:74cf/64 Scope:Link 

UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 

RX packets:528 errors:0 dropped:0 overruns:0 frame:0 
TX packets:333 errors:0 dropped:0 overruns:0 carrier:0 
collisions:0 txqueuelen:1000 

RX bytes:45429 (44.3 KiB) TX bytes:48763 (47.6 KiB) 


eth0:0 Link encap:Ethernet  HWaddr 08:00:27:f£5:74:cf 
Inet addr: 10L OA Ssn39 Beast Osos. Eb A MASIA ZO 90r) 
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 


lo Link encap:Local Loopback 

inet addr:127.0.0.1 Mask:255.0.0.0 

inet6 addr: ::1/128 Scope:Host 

UP LOOPBACK RUNNING MTU:65536 Metric:1 

RX packets:0 errors:0 dropped:0 overruns:0 frame:0 
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 
collisions:0 txqueuelen:0 

RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) 


root@server42:~# 
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4.6. Debian package management 


To get all information about the newest packages form the online repository: 


root@server42:~# aptitude update 
Get: 1 http://ftp.be.debian.org jessie InRelease [191 kB] 
Get: 2 http://security.debian.org jessie/updates InRelease [84.1 kB] 
Get: 3 http://ftp.be.debian.org jessie-updates InRelease [117 kB] 
Get: 4 http://ftp.be.debian.org jessie-backports InRelease [118 kB] 
5 
6 
O 


Get: http://security.debian.org jessie/updates/main Sources [14 B] 
Get: http://ftp.be.debian.org jessie/main Sources/DiffIndex [7,876 B] 
(output truncated) 


To download and apply all updates for all installed packages: 


root@server42:~# aptitude upgrade 
Resolving dependencies... 
The following NEW packages will be installed: 
firmware-linux-free(a) irqbalance{a} libnumal{a} linux-image-3.16.0-4-amd64(a) 
The following packages will be upgraded: 
busybox file libc-bin libcé libexpatl1 libmagicl libpaper-utils libpaperl libsqlite3-0 
linux-image-amd64 locales multiarch-support 
12 packages upgraded, 4 newly installed, 0 to remove and 0 not upgraded. 
Need to get 44.9 MB of archives. After unpacking 161 MB will be used. 
Do you want to continue? [Y/n/?] 
(output truncated) 


To install new software (vim and tmux in this example): 


root@server42:~# aptitude install vim tmux 
The following NEW packages will be installed: 
tmux vim vim-runtime{a} 
O packages upgraded, 3 newly installed, 0 to remove and 0 not upgraded. 
Need to get 6,243 kB of archives. After unpacking 29.0 MB will be used. 
Do you want to continue? [Y/n/?] 
Get: 1 http://ftp.be.debian.org/debian/ jessie/main tmux amd64 1.9-6 [245 kB] 
Get: 2 http://ftp.be.debian.org/debian/ jessie/main vim-runtime all 2:7.4.488-1 [5,046 kB] 
Get: 3 http://ftp.be.debian.org/debian/ jessie/main vim amd64 2:7.4.488-1 [952 kB] 


Refer to the package management chapter in LinuxAdm.pdf for more information. 
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Chapter 5. installing CentOS 7 


This module is a step by step demonstration of an actual installation of CentOS 7. 


We start by downloading an image from the internet and install CentOS 7 as a virtual 
machine in Virtualbox. We will also do some basic configuration of this new machine like 
setting an ip address and fixing a hostname. 


This procedure should be very similar for other versions of CentOS, and also for 
distributions like RHEL (Red Hat Enterprise Linux) or Fedora. This procedure can also be 
helpful if you are using another virtualization solution. 
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5.1. download a CentOS 7 image 


This demonstration uses a laptop computer with Virtualbox to install CentOS 7 as a virtual 
machine. The first task is to download an .iso image of CentOS 7. 


The CentOS 7 website looks like this today (November 2014). They change the look 
regularly, so it may look different when you visit it. 


Download CentOS - Iceweasel 


File Edit View History Bookmarks Tools Help 
e www.centos.org/download C )uckDuckGo avs t ©- = 


| #8 Download CentOS x | dh | 


Se CentOS «c 


Download CentOS. 


ould like to remind you that the primary m 
i contributions by users such as yourself 


ikes money and resource 


DVD ISO Need a Cloud Image? 
oking to run CentOS in the Cloud? We 


Everything ISO we a number of 


s are also available 


You can download a full DVD, which allows for an off line installation of a graphical 
CentOS 7 desktop. You can select this because it should be easy and complete, and should 
get you started with a working CentOS 7 virtual machine. 
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But I clicked instead on 'alternative downloads', selected CentOS 7 and x86 64 and ended 
up on a mirror list. Each mirror is a server that contains copies of CentOS 7 media. I 
selected a Belgian mirror because I currently am in Belgium. 


d Index of /centos/7.0.1406/isos/x86 64 - Iceweasel a oo 5: 
File Edit View History Bookmarks Tools Help 
€ € or.eurid.eu/centos/7.0.1406/isos/x86_64 Y S| |@v DuckDuckG avs tA E 


| Index of /centos/7.0.1406/... x | db | 


Index of /centos/7.0.1406/isos/x86 64 


Name Last modified Size Description 
«9 Parent Director > 
[El o README. txt 21-Jul-2014 09:24 2.6K 
[Es] Cent0S-7.0-1406-x86 64-DVD.iso 06-Jul-2014 19:33 3.9G 
£] Cent0S-7.0-1406-x86 64-DVD.torrent 07-Jul-2014 14:16 155K 
A Cent0S-7.0-1406-x86 64-Everything.iso 05-Jul-2014 00:16 6.66 
(9j Cent0S-7.0-1406-x86 64-Everything.torrent 07-Jul-2014 14:16 264K 
£] Cent0S-7.0-1406-x86 64-GnomeLive.iso 04-Jul-2014 19:22 1.0G 
ls] Cent0S-7.0-1406-x86 64-Gnomelive.torrent 07-Jul-2014 14:16 42K 
A Cent0S-7.0-1406-x86 64-KdeLive.iso 04-Jul-2014 19:44 1.2G 
ea) Cent0S-7.0-1406-x86_64-KdeLive. torrent 07-Jul-2014 14:16 49K 
£] Cent0S-7.0-1406-x86 64-livecd.iso 04-Jul-2014 19:00 687M 
la] Cent0S-7.0-1406-x86 64-livecd.torrent 07-Jul-2014 14:16 27K 
P) cent0S-7.0-1406-x86 64-Minimal.iso 17-Jul-2014 16:16 566M 
P) cent0S-7.0-1406-x86 64-NetInstall.iso 04-Jul-2014 17:59 362M 
[2] CentOS-7.0-1406-x86 64-NetInstall.torrent 07-Jul-2014 14:16 15K 
Bl md5sum.txt 21-Jul-2014 09:24 486 
E mdSsum. txt. asc 21-Jul-2014 09:24 1.3K 
E] shalsum. txt 21-Jul-2014 09:24 542 
E] shalsum.txt.asc 21-Jul-2014 09:24 1.4K 
[El sha256sum. txt 21-Jul-2014 09:24 1.3K 
=] sha256sum.txt.asc 21-Jul-2014 09:24 2.2K 


There is again the option for full DVD's and more. This demonstration will use the minimal 
so file, because it is much smaller in size. The download takes a couple of minutes. 


Library 
da ® Organize* Clear Downloads Search Downloac (à 
> O History CentOS-7.0-1406-x86 64-Minimal.iso 
J Downloads í j 
> Tags 


7 minutes remaining — 278 of 566 MB (596 KB/sec) 
> E All Bookmarks 


Verify the size of the file after download to make sure it is complete. Probably a right click 
on the file and selecting 'properties' (if you use Windows or Mac OSX). 


I use Linux on the laptop already: 


paul@debian8:~$ ls -lh CentOS-7.0-1406-x86 64-Minimal.iso 
-rw-r--r-- 1 paul paul 566M Nov 1 14:45 CentOS-7.0-1406-x86 64-Minimal.iso 


Do not worry if you do no understand the above command. Just try to make sure that the 
size of this file is the same as the size that is mentioned on the CentOS 7 website. 
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5.2. Virtualbox 


This screenshot shows up when I start Virtualbox. I already have four virtual machines, you 
might have none. 


E Oracle VM VirtualBox Manager *-DX 
File Machine Help 


C T a v (3 Details (8) Snapshots 


New Settings Start Discard 


O debian7 [El General [El Preview = 
- Powered Off 
9 Name: RHEL7 
EN winxp Operating System: Red Hat (64 bit) 
@ Powered Off syatam 
RHEL7 RS Base Memory 544 MB 
(O) Powered Off Processors: 2 
Execution Cap: 90% 
(55) centos7 Boot Order: Hard Disk, CD/DVD 
[ ` @ Powered off Acceleration: VT-WAMDV, Nested Paging, 
PAE/NX 
Display 
Video Memory: 32 MB 
Remote Desktop Server: Disabled 
Video Capture: Disabled 


Storage 


Controller: IDE 

IDE Secondary Master: [CD/DVD] rhel-server-7.0-x86_64-dvd.iso (3.49 GB) 
Controller: SATA 

SATA Port 0: RHEL7.vdi (Normal, 40.00 GB) 


Qa Audio m 


Below are the steps for creating a new virtual machine. Start by clicking New and give your 
machine a name (I chose server33). Click Next. 


Create Virtual Machine 
Name and operating system 


Please choose a descriptive name for the new 
virtual machine and select the type of operating 
system you intend to install on it. The name you 
choose will be used throughout VirtualBox to 
identify this machine. 


Name: [server33| 
Type: [Linux -] 
l e. 
Version: [Red Hat (64 bit) -] i 


Hide Description | < Back || vex» | Cancel | 
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A Linux computer without graphical interface will run fine on half a gigabyte of RAM. 


Create Virtual Machine 


Memory size 


Select the amount of memory (RAM) in megabytes 
to be allocated to the virtual machine. 


The recommended memory size is 512 MB. 


A ot utenti ESESRRER ESSE Be 


4 MB 8192 MB 


< Back | [tex > | Cancel | 


A Linux virtual machine will need a virtual hard drive. 


Create Virtual Machine 


Hard drive 


If you wish you can add a virtual hard drive to the 
new machine. You can either create a new hard 
drive file or select one from the list or from another 
location using the folder icon. 


If you need a more complex storage set-up you can 
skip this step and make the changes to the 
machine settings once the machine is created. 


The recommended size of the hard drive is 8.00 
GB. 


© Do not add a virtual hard drive 


C Use an existing virtual hard drive file 


[RHEL.vdi (Normal, 40,00 GB) za 


< Back | [_ create | Cancel | 
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Any format will do for our purpose, so I left the default vdi. 


Create Virtual Hard Drive 


Hard drive file type 


Please choose the type of file that you would like to use for 
the new virtual hard drive. If you do not need to use it with 
other virtualization software you can leave this setting 
unchanged. 


a VMDK (Virtual Machine Disk) 
C VHD (Virtual Hard Disk) 

C HDD (Parallels Hard Disk) 

C QED (QEMU enhanced disk) 
C QCOW (QEMU Copy-On-Write) 


Hide Description | < Back | [tex > | Cancel | 


The default dynamically allocated type will save disk space (until we fill the virtual disk 
up to 100 percent). It makes the virtual machine a bit slower than fixed size, but the fixed 
size speed improvement is not worth it for our purpose. 


Create Virtual Hard Drive 
Storage on physical hard drive 


Please choose whether the new virtual hard drive file should 
grow as it is used (dynamically allocated) or if it should be 
created at its maximum size (fixed size), 


A dynamically allocated hard drive file will only use space on 
your physical hard drive as it fills up (up to a maximum fixed 
size), although it will not shrink again automatically when 
space on it is freed. 


A fixed size hard drive file may take longer to create on some 
systems but is often faster to use. 


C Fixed size 


< Back | [tex > | Cancel | 
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The name of the virtual disk file on the host computer will be server33.vdi in my case (I left 
it default and it uses the vm name). Also 16 GB should be enough to practice Linux. The 
file will stay much smaller than 16GB, unless you copy a lot of files to the virtual machine. 


Create Virtual Hard Drive 


File location and size 


Please type the name of the new virtual hard drive file into the 
box below or click on the folder icon to select a different folder 
to create the file in. 


[server33 la 


Select the size of the virtual hard drive in megabytes. This size 
is the limit on the amount of file data that a virtual machine will 
be able to store on the hard drive. 


J———— [ [6.00 GB 


2.00 TB 


< Back | | create | Cancel | 


You should now be back to the start screen of Virtualbox. If all went well, then you should 


see the machine you just created in the list. 


EN Oracle VM VirtualBox Manager 
File Machine Help 


®t- OX 


New Settings Start Discard 


SE debian7 [El General [El Preview 
@ Powered Off 
Name: server33 
PR winxp Operating System: Red Hat (64 bit) 
sh @ Powered Off system 
e RHEL7 Base Memory 512 MB 
@ Powered Off Boot Order: CD/DVD, Hard Disk 
Acceleration: | VT AMD V, Nested Paging, 
64 centos7 PAE/NX 
Y @ Powered Off 


Display 

Video Memory: 12 MB 
Remote Desktop Server: Disabled 
Video Capture: Disabled 


Storage 
Controller: IDE 


Controller: SATA 
SATA Port 0: server33.vdi (Normal, 16.00 GB) 


Qa Audio 
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C e » Y (3 Details (8j Snapshots 


IDE Secondary Master: [CD/DVD] CentOS-7.0-1406-x86 64-Minimal.iso (566.00 MB) 
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After finishing the setup, we go into the Settings of our virtual machine and attach the .iso 
file we downloaded before. Below is the default screenshot. 


server33 - Settings * 0x 
Storage 
Storage Tree —————————— | Attributes 
Name: |IDE 
QR Audio © Empty Type: [Puxa -] 
¿4 Controller: SATA Jv Use Host I/O Cache 


(E Network 
(9 Serial Ports 


@ USB 


Shared Folders 


server33.vdi 


server33 - Settings * 0x 
Storage 
Storage Tree —————————————— | Attributes 


CD/DVD Drive: [IDE Secondary M: y] Q 


& Controller: IDE 


Ba Audio O CentOS-7.0-1406-x86 6... I Live CD/DVD 
E Network & Controller: SATA Information 
server33.vdi Type: Image 


(9 Serial Ports 


@ USB 


Shared Folders 


Size: 566.00 MB 
Location: /home/paul/Download... 
Attachedto: -- 


aec 


Cancel | Help | 
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5.3. CentOS 7 installing 


The screenshots below will show every step from starting the virtual machine for the first 
time (with the .iso file attached) until the first logon. 


You should see this when booting, otherwise verify the attachment of the .iso file form the 
previous steps. Select Test this media and install CentOS 7. 


E server33 [Running] - Oracle VM VirtualBox SX 


Machine View Devices Help 


Install CentOS 7 
Test this media & install CentOS 7 


Troubleshooting 


fiutomatic boot in 59 seconds... 


8d O Y Lía QU | C9 (€) Left winkey 
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Carefully select the language in which you want your CentOS. I always install operating 
systems in English, even though my native language is not English. 


Also select the right keyboard, mine is a US qwerty, but yours may be different. 


E 


4 server33 [Running] - Oracle VM VirtualBox ^-0X 
Machine View Devices Help 


CENTOS 7 INSTALLATION 
Bus 


CentOS WELCOME TO CENTOS 7. 


What Language would you like to use during the installation process? 


English English > | English (United States) 
| 


| English (United Kingdom) | 


Afrikaans Afrikaans 
. English (India) 
Amic Amharic : 
English (Australia) 
a | Arabic 
Asus) Arabi English (Canada) 
biha Assamese English (Denmark) 
Asturianu Asturian English (Ireland) 
Benapyckaa — Belarusikn English (New Zealand) 


b»arapcka Bulgarian English (Nigeria) 


ale Bengali English (Hong Kong SAR China) 
English (Philippines) 
Bosanski Bosnian El ii 
English (Singapore) 
Carls Catalan English (South Africa) 


Cesta Czech English (Zambia) 
Cymraeg Welsh English (Zimbabwe) 
Dansk | Danish English (Botswana) y 
Typ a 


Quit Continue 


BOF Bm Y | C9 () Left winkey 


You should arrive at a summary page (with one or more warnings). 


5 server33 [Running] - Oracle VM VirtualBox *-D0X 
Machine View Devices Help 
INSTALLATION SUMMARY CENTOS 7 INSTALLATION 
Bus 
CentOS LOCALIZATION 
DATE & TIME KEYBOARD 
Americas/New York timezone English (US) 


LANGUAGE SUPPORT 
English (United States) 


SOFTWARE 
INSTALLATION SOURCE SOFTWARE SELECTION 
Local media Minimal Install 

SYSTEM 
INSTALLATION DESTINATION NETWORK & HOSTNAME 
Automatic partitioning selected Not connected 


Quit 


BOF Mm Y | O (9) Left winkey 
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Start by configuring the network. During this demonstration I had a DHCP server running 
at 192.168.1.42, yours is probably different. Ask someone (a network administator ?) for 
help if this step fails. 


E server33 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


NETWORK & HOSTNAME 


Ethernet (enpOs3) 
aM oO 
Hardware Address 08:00:27:1C:F5:AB 
Speed 1000 Mb/s 
IP Address 10.0.2.15 
Subnet Mask 255.255.255.0 
Default Route 10.0.2.2 


DNS 192.168.1.42 


uu - Configure... 


Hostname: | localhost.localdomain 


(g C e? ciu | 9 (S)teft winkey 4 


Select your time zone, and activate ntp. 


E server33 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


DATE & TIME 


Done 


Region: | Europe v | City: Brussels 


^I ^I ^ 


17:40 PM Cres November v | 1 v | 2014v 
v v v 


B C) Y O | O (e) Left winkey 
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Choose a mirror that is close to you. If you can't find a local mirror, then you can copy the 
one from this screenshot (it is a general CentOS mirror). 


server33 [Running] - Oracle VM VirtualBox ^-nX 


Machine View Devices Help 


INSTALLATION SOURCE CENTOS 7 INSTALLATION 


Which installation source would you like to use? 


Auto-detected installation media 


Ó On the network 


http:// Y | | mirror.centos.org/centos/7/05/x86-64/ Proxy setup.. 


This URL refers to a mirror list k 
Additional repositories 


Enabled Name 


BOF Om GO| O (9) Left winkey 


It can take a couple of seconds before the mirror is verified. 


E server33 [Running] - Oracle VM VirtualBox «e HX 
Machine View Devices Help 
INSTALLATION SUMMARY CENTOS 7 INSTALLATION 
Bus 
LOCALIZATION 
DATE & TIME KEYBOARD 
Europe/Brussels timezone English (US) 


LANGUAGE SUPPORT 
English (United States) 


ISOFTWARE 


INSTALLATION SOURCE SOFTWARE SELECTION 
http://mirror.centos.org/centos/7/0s/x86_64/ A Source changed - please verify 
k 
ISYSTEM 
INSTALLATION DESTINATION NETWORK & HOSTNAME 
Automatic partitioning selected Wired (enpOs3) connected 
Quit 


BOF mw Y | O (9) Left winkey 
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I did not select any software here (because I want to show it all in this training). 


server33 [Running] - Oracle VM VirtualBox ^-nX 


Machine View Devices Help 


SOFTWARE SELECTION CENTOS 7 INSTALLATION 
Base Environment Add-Ons for Selected Environment 
© L Install | Compatibility Libraries 
Basic functionality Compatibility libraries for applications built on previous versions of 
Infrastructure Server | CentOS Linux 


Server for operating network infrastructure services. 
Development Tools 

File and Print Server A basic development environment 

File, print, and storage server for enterprises Smart Card Support 

Support for using smart card authentication. 
Basic Web Server 

Server for serving static and dynamic internet content. 


Virtualization Host 
Minimal virtualization host. 


Server with GUI 
Server for operating network infrastructure services, with a GUI. 


GNOME Desktop 
GNOME is a highly intuitive and user friendly desktop environment 


KDE Plasma Workspaces k 
The KDE Plasma Workspaces, a highly-configurable graphical user 
interface which includes a panel, desktop, system icons and desktop 


widgets, and many powerful KDE applications. 


Development and Creative Workstation 
Workstation for software, hardware, graphics, or content 


development y 


B O P a a Y | C9 () Left winkey 


After configuring network, location, software and all, you should be back on this page. Make 
sure there are no warnings anymore (and that you made the correct choice everywhere). 


a server33 [Running] - Oracle VM VirtualBox AAK 
Machine View Devices Help 
INSTALLATION SUMMARY CENTOS 7 INSTALLATION 
Bus 
ie | OCALIZATION 
DATE & TIME KEYBOARD 
Europe/Brussels timezone English (US) 
LANGUAGE SUPPORT 


English (United States) 


SOFTWARE 
INSTALLATION SOURCE SOFTWARE SELECTION 
http://mirror.centos.org/centos/7/0s/x86_64/ Minimal Install 
k 
ISYSTEM 
INSTALLATION DESTINATION NETWORK & HOSTNAME 
Automatic partitioning selected Wired (enpOs3) connected 
Quit Begin Installation 


BOX wa Y | C9 (€) Left winkey 
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You can enter a root password and create a user account while the installation is 
downloading from the internet. This is the longest step, it can take several minutes (or up to 


an hour if you have a slow internet connection). 


^-nDnDX 


server33 [Running] - Oracle VM VirtualBox 


CENTOS 7 INSTALLATION 
Bus 


E 
Machine View Devices Help 
CONFIGURATION 


ie USER SETTINGS 
@ USER CREATION 


ROOT PASSWORD 
Root password is set A User paul will be created 


“ Starting package installation process 


CentOS Core SIG 


Produces the CentOS Linux Distribution. 


wiki.centos.org/SpecialinterestGroup 
(gl Cà dB (1 O | 9 E Left Winkey 


If you see this, then the installation was successful. 


Time to reboot the computer and start CentOS 7 for the first time. 


t- 0X 


server33 [Running] - Oracle VM VirtualBox 


CENTOS 7 INSTALLATION 
Bus 


[: 


Machine View Devices Help 
CONFIGURATION 


4 | 


CentOS 


Complete! 
CentOS is now successfully installed on your system and ready for you to use! Go ahead and reboot to start using it! 
Reboot 


BOF Mite QU | C9 (€) Left winkey 
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This screen will appear briefly when the virtual machines starts. You don't have to do 
anything. 


E server33 [Running] - Oracle VM VirtualBox 
Machine View Devices Help 


CentOS Linux, with Linux 3.10.0-123.e17.x86 64 


CentOS Linux, with Linux O-rescue-5a89376246d640eabbf ee196a8d4376e9 


Use the f and 1 keys to change the selection. 
Press 'e' to edit the selected item, or 'c' for a command prompt. 
The selected entry will be started automatically in 5s. 


GP (i umi Y | C9 (€) Left winkey 


After a couple of seconds, you should see a logon screen. This is called a tty or a getty. Here 


you can type root as username. The login process will then ask your password (nothing will 
appear on screen when you type your password). 


E server33 [Running] - Oracle VM VirtualBox t- 0X 
Machine View Devices Help 


CentOS Linux ? (Core) 
Kernel 3.18.8-123.6e17.x86 64 on an x86 64 


localhost login: _ 


GP d umi (Y [ C9 (€) Left winkey 
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And this is what it looks like after logon. You are logged on to your own Linux machine, 
very good. 


E server33 [Running] - Oracle VM VirtualBox t. 0X 


Machine View Devices Help 


CentOS Linux ? (Core) 
Kernel 3.18.8-123.617.x86 64 on an x86 64 


localhost login: root 
Password: 
[Iroot8 localhost “14 _ 


P 1 wai Q | (9 (€) Left winkey 


All subsequent screenshots will be text only, no images anymore. 


For example this screenshot shows three commands being typed on my new CentOS 7 
install. 


[root@localhost ~]# who am i 
root pts/0 Zona quoq ALA! 
[root@localhost ~]# hostname 


localhost.localdomain 
[root@localhost ~]# date 
Sat Nov 1 22:4 3 CET 2014 


When using ssh the same commands will give this screenshot: 


[root@localhost ~]# who am i 

root pts/0 2014-11-01 21:00 (192.168.1.35) 
[root@localhost ~]# hostname 

localhost.localdomain 

[root@localhost ~]# date 

Sat Nov 1 22:10:04 CET 2014 

[root@localhost ~]# 


If the last part is a bit too fast, take a look at the next topic CentOS 7 first logon. 
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5.4. CentOS 7 first logon 


All you have to log on, after finishing the installation, is this screen in Virtualbox. 


E server33 [Running] - Oracle VM VirtualBox * 


Machine View Devices Help 


CentOS Linux ? (Core) 
Kernel 3.18.8-123.6e17.x86 64 on an x86 64 


localhost login: _ 


P o uai Y | (9 (€) Left winkey 


This is workable to learn Linux, and you will be able to practice a lot. But there are more 
ways to access your virtual machine, the next chapters discuss some of these and will also 
introduce some basic system configuration. 


5.4.1. setting the hostname 
Setting the hostname is a simple as changing the /etc/hostname file. As you can see here, 


it is set to localhost.localdomain by default. 


[root@localhost ~]# cat /etc/hostname 
localhost.localdomain 


You could do echo server33.netsec.local > /etc/hostname followed by a reboot. But there 
is also the new CentOS 7 way of setting a new hostname. 


[root@localhost ~]# nmtui 


The above command will give you a menu to choose from with a set system hostname 
option. Using this nmtui option will edit the /etc/hostname file for you. 


[root@localhost ~]# cat /etc/hostname 
server33.netsec.local 

[root@localhost ~]# hostname 
server33.netsec.local 

[root@localhost ~]# dnsdomainname 
netsec.local 


For some reason the documentation on the centos.org and docs.redhat.com websites tell 
you to also execute this command: 


[root@localhost ~]# systemctl restart systemd-hostnamed 
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5.5. Virtualbox network interface 


By default Virtualbox will connect your virtual machine over a nat interface. This will 
show up as a 10.0.2.15 (or similar). 


[root@server33 ~]# ip a 
1: lo: «LOOPBACK,UP,LOWER UP» mtu 65536 qdisc noqueue state UNKNOWN 
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 
inet 127.0.0.1/8 scope host lo 
valid lft forever preferred lft forever 
inet6 ::1/128 scope host 
valid lft forever preferred lft forever 
2: enp0s3: «BROADCAST,MULTICAST,UP,LOWER UP» mtu 1500 qdisc pfifo fast s\ 
tate UP qlen 1000 
Iink/ether 08:00:27: o: f5rab bra Ef me: restr et ff 
inet 10.0.2.15/24 5rd 10.0.2.255 scope global dynamic enplss 
valid lft 86399sec preferred lft 86399sec 
inet6 fe80::a00:27ff:felc:f5ab/64 scope link 
valid lft forever preferred lft forever 


You can change this to bridge (over your wi-fi or over the ethernet cable) and thus make it 
appear as if your virtual machine is directly on your local network (receiving an ip address 
from your real dhcp server). 


A server33 - Settings tox 


Serial Ports Y Advanced NAT Network 


USB 
Shared Folders 


Internal Network 
Host-only Adapter 
Generic Driver 


Adapter Type: ktop (82540EM) 


[E] General | Network 
System 
Display Adapter 1 | Adapter 2 | Adapter 3 | Adapter 4 | 
Storage Jv Enable Network Adapter 
Audio Attached to: |Bridged Adapter 
A g p 

N . Not attached 

BEEN NAT 
e 
8 


Promiscuous Mode: 


MAC Address; [0800271CF5AB (5) 


|v Cable Connected 


Port Fonwarding | 


Cancel | Help | 


You can make this change while the vm is running, provided that you execute this command: 


[root@server33 ~]# systemctl restart network 
[root@server33 ~]# ip a s dev enp0s3 
2: enp0s3: «BROADCAST,MULTICAST,UP,LOWER UP» mtu 1500 qdisc pfifo fast s\ 
tate UP qlen 1000 
lank/ether OS 002 lero abbr fi: PETE EEE et: E 
inet 192.168.1.110/24 brd 192.168.1.255 scope global dynamic enp0s3 
valid_lft 7199sec preferred_lft 7199sec 
inet6 fe80::a00:27ff:felc:f5ab/64 scope link 
valid_lft forever preferred_lft forever 
[root@server33 ~]# 


53 


installing CentOS 7 


5.6. configuring the network 


The new way of changing network configuration is through the nmtui tool. If you want to 
manually play with the files in /etc/sysconfig/network-scripts then you will first need to 
verify (and disable) NetworkManager on that interface. 


Verify whether an interface is controlled by NetworkManager using the nmcli command 
(connected means managed bu NM). 


[root@server33 ~]# nmcli dev status 
DEVICE TYPE STATE CONNECTION 
enp0s3 ethernet connected  enp0s3 

To loopback unmanaged -- 


Disable NetworkManager on an interface (enpOs3 in this case): 


echo 'NM CONTROLLED-no' >> /etc/sysconfig/network-scripts/ifcfg-enp0s3 


You can restart the network without a reboot like this: 


[root@server33 ~]# systemctl restart network 


Also, forget ifconfig and instead use ip a. 
[root@server33 ~]# ip a s dev enp0s3 | grep inet 
inet 192.168.1.110/24 brd 192.168.1.255 scope global dynamic enp0s3 


inet6 fe80::a00:27ff:felc:f5ab/64 scope link 
[root@server33 ~]# 


5.7. adding one static ip address 


This example shows how to add one static ip address to your computer. 


[root@server33 ~]# nmtui edit enp0s3 


In this interface leave the IPv4 configuration to automatic, and add an ip address just below. 


IPv4 CONFIGURATION <Automatic> <Hide> 
Addresses 10.104.33.32/16 <Remove> 


Execute this command after exiting nmtui. 


[root@server33 ~]# systemctl restart network 


And verify with ip (not with ifconfig): 


[root@server33 ~]# ip a s dev enp0s3 | grep inet 
inet 192.168.1.110/24 brd 192.168.1.255 scope global dynamic enp0s3 
inet 10.104.33.32/16 brd 10.104.255.255 scope global enp0s3 
inet6 fe80::a00:27ff:felc:f5ab/64 scope link 

[root@server33 ~]# 
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5.8. package management 


Even with a network install, CentOS 7 did not install the latest version of some packages. 
Luckily there is only one command to run (as root). This can take a while. 


[root@server33 ~]# yum update 
Loaded plugins: fastestmirror 
Loading mirror speeds from cached hostfile 
* base: centos.weepeetelecom.be 
* extras: centos.weepeetelecom.be 
* updates: centos.weepeetelecom.b 
Resolving Dependencies 
--» Running transaction check 
---» Package NetworkManager.x86 64 1:0.9.9.1-13.git20140326.4dba720.e17 \ 
will be updated 
(output truncated) 


You can also use yum to install one or more packages. Do not forget to run yum update 
from time to time. 


[root@server33 ~]# yum update -y && yum install vim -y 
Loaded plugins: fastestmirror 
Loading mirror speeds from cached hostfile 
* base: centos.weepeetelecom.be 
(output truncated) 


Refer to the package management chapter for more information on installing and removing 
packages. 
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5.9. logon from Linux and MacOSX 


You can now open a terminal on Linux or MacOSX and use ssh to log on to your virtual 
machine. 


paul@debian8:~$ ssh root@192.168.1.110 
root@192.168.1.110's password: 
East Login: Sun Now 2 41:53:59] 2014 


[root@server33 ~]# hostname 
server33.netsec.local 
[root@server33 ~]# 


5.10. logon from MS Windows 


There is no ssh installed on MS Windows, but you can download putty.exe from http:// 
www.chiark.greenend.org.uk/~sgtatham/putty/download.html (just Google it). 


Use putty.exe as shown in this screenshot (I saved the ip address by giving it a name 
'server33' and presing the 'save' button). 


i& PuTTY Configuration 
Category: 


=) Session Basic options for your PuTTY session 
Logging 

=). Terminal 
Keyboard Host Name (or IP address] Port 


Bel 192.168.1.110 |[22 


Features Connection type: 

E Window | OhRaw OTelnet O Rlogn (9 SSH © Serial 
Appearance 
Behaviour 
Translation Saved Sessions 
Selection serverda 
Colours Default Settings 

& Connection server33 
Data 
Proxy 
Telnet 
Rlogin 

SSH 
Serial 


Specify the destination you want to connect to 


Load, save or delete a stored session 


Close window on exit: 
O álways O Never © Only on clean exit 
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The first time you will get a message about keys, accept this (this is explained in the ssh 
chapter). 


E 192.168.1.110 - PuTTY 


Pul TY Security Alert 


The server's host key is not cached in the registry. You 
have no guarantee that the server is the computer you 
think it is. 

The server's rsa2 key Fingerprint is: 

ssh-rsa 2048 07:5b:6c:62:3F:49:5f:8b:0f:3b:84:b6:8b:56:58:c3 
If you trust this host, hit Yes to add the key to 

PuTTY's cache and carry on connecting. 

If you want to carry on connecting just once, without 
adding the key to the cache, hit No. 

If you do not trust this host, hit Cancel to abandon the 
connection. 


Enter your userid (or root) and the correct password (nothing will appear on the screen when 
typing a password). 


x? rooteserver33:- 
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Chapter 6. getting Linux at home 


This chapter shows a Ubuntu install in Virtualbox. Consider it legacy and use CentOS7 
or Debian8 instead (each have their own chapter now). 


This book assumes you have access to a working Linux computer. Most companies have 
one or more Linux servers, if you have already logged on to it, then you 're all set (skip this 
chapter and go to the next). 


Another option is to insert a Ubuntu Linux CD in a computer with (or without) Microsoft 
Windows and follow the installation. Ubuntu will resize (or create) partitions and setup a 
menu at boot time to choose Windows or Linux. 


If you do not have access to a Linux computer at the moment, and if you are unable or unsure 
about installing Linux on your computer, then this chapter proposes a third option: installing 
Linux in a virtual machine. 


Installation in a virtual machine (provided by Virtualbox) is easy and safe. Even when you 
make mistakes and crash everything on the virtual Linux machine, then nothing on the real 
computer is touched. 


This chapter gives easy steps and screenshots to get a working Ubuntu server in a Virtualbox 
virtual machine. The steps are very similar to installing Fedora or CentOS or even Debian, 
and if you like you can also use VMWare instead of Virtualbox. 
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6.1. download a Linux CD image 


Start by downloading a Linux CD image (an .ISO file) from the distribution of your choice 
from the Internet. Take care selecting the correct cpu architecture of your computer; choose 
1386 if unsure. Choosing the wrong cpu type (like x86 64 when you have an old Pentium) 
will almost immediately fail to boot the CD. 


Home Ubuntu Business Cloud TV Download Support Project 


Ubuntu Ubuntu Server 


Download Ubuntu Server 


ubuntu? 


Community Partners Shop 


Type to search ) 


YY Tweet 228 KiLike © 2k 


You can download Ubuntu Server now - it's completely Free. 


Download 


Buy CDs 


Ubuntu Server for ARM 


Download Ubuntu Server 


Click the big orange button to 
download the latest version of 
Ubuntu. You will need to create a 
CD or USB stick to install Ubuntu. 


Our long-term support (LTS) 
releases are supported for five 
years on the server. Perfect for 
organisations that need more 
stability for larger deployments. 


Download options 


| Ubuntu 11.10 - Latest version 


Start download 


[ -bit - 
€— Ubuntu Server 11.10 


64-bit 


Direct url for this download 


6.2. download Virtualbox 


Step two (when the .ISO file has finished downloading) is to download Virtualbox. If you are 
currently running Microsoft Windows, then download and install Virtualbox for Windows! 


About 


Screenshots 


Downloads 


<P 


Download VirtualBox 


Here, you will find links to VirtualBox binaries and its source code. 


Documentation 


End-user docs 


Technical docs 


Contribute 


VirtualBox binaries 


By downloading, you agree to the terms and conditions of the respectivi 


* VirtualBox platform packages. The binaries are released under 
o VirtualBox 4.1.8 for Windows hosts © x86/amd64 
o VirtualBox 4.1.8 for OS X hosts ^ x86/amd64 
o VirtualBox 4.1.8 for Linux hosts 


o VirtualBox 4.1.8 for Solaris hosts > x86/amd64 
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6.3. create a virtual machine 


Now start Virtualbox. Contrary to the screenshot below, your left pane should be empty. 


A a y 


Oracle VM VirtualBox Manager C 


ay € À 
Q v FETU a sranna: 


New Settings Start Discard 


4 A oe ot 


© — - - 


Led _ 
Pu — 


pr E 
E e - 
"d 

a 


y Tm a» a = 


— y a i + 
- = OD WI PPS z 


Click New to create a new virtual machine. We will walk together through the wizard. The 
screenshots below are taken on Mac OSX; they will be slightly different if you are running 
Microsoft Windows. 


Create New Virtual Machine 


Welcome to the New Virtual Machine Wizard! 


This wizard will guide you through the steps that are necessary to create a new 
virtual machine for VirtualBox. 


Use the Continue button to go to the next page of the wizard and the Go Back 
button to return to the previous page. You can also press Esc if you want to 
cancel the execution of this wizard. 


Ai 
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Name your virtual machine (and maybe select 32-bit or 64-bit). 


menn Create New Virtual Machine 


VM Name and OS Type 


Enter a name for the new virtual machine and select the type of the guest 
operating system you plan to install onto the virtual machine. 


The name of the virtual machine usually indicates its software and hardware 
configuration. It will be used by all VirtualBox components to identify your 
virtual machine. 


Name 


mijnvirtuelemachine 


_OS Type. 


Operating System: | Linux $4 BA 
Version: | Ubuntu (64 bit) 


( Go Back ( Continue ) 


Mi 


Give the virtual machine some memory (512MB if you have 2GB or more, otherwise select 
256MB). 


000 Create New Virtual Machine 


Memory 


Select the amount of base memory (RAM) in megabytes to be allocated to the 
virtual machine. 


The recommended base memory size is 512 MB. 
Base Memory Size 


=I o S12 v 


4 MB 8192 MB 


( Go Back ( Continue 


"A 
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Select to create a new disk (remember, this will be a virtual disk). 


Create New Virtual Machine 


Virtual Hard Disk 


If you wish you can now add a start-up disk to the new machine. You can either 
create a new virtual disk or select one from the list or from another location 
using the folder icon. 


If you need a more complex virtual disk setup you can skip this step and make 
the changes to the machine settings once the machine is created. 


The recommended size of the start-up disk is 8,00 GB. 
(Vi Start-up Disk 
(*) Create new hard disk 
O Use existing hard disk 
Q CentOS6.vdi (Normal, 16,00 GB) +1 i 


( Continue ) 


PA 


If you get the question below, choose vdi. 


Create New Virtual Disk 


Welcome to the virtual disk creation wizard 


This wizard will help you to create a new virtual disk for your 
virtual machine. 


Use the Continue button to go to the next page of the wizard and 
the Go Back button to return to the previous page. You can also 
press Esc if you want to cancel the execution of this wizard. 


Please choose the type of file that you would like to use for the 
P | new virtual disk. If you do not need to use it with other 
; virtualization software you can leave this setting unchanged. 


File type — .— — 


(*) VDI (VirtualBox Disk Image) 
(O) VMDK (Virtual Machine Disk) 
© VHD (Virtual Hard Disk) 

©) HDD (Parallels Hard Disk) 


Go Back | ( Continue ) 


h 
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Choose dynamically allocated (fixed size is only useful in production or on really old, slow 
hardware). 


Create New Virtual Disk 


Virtual disk storage details 


Please choose whether the new virtual disk file should be allocated 
as it is used or if it should be created fully allocated. 


A dynamically allocated virtual disk file will only use space on 
your physical hard disk as it fills up, although it will not shrink 
again automatically when space on it is freed. 


A fixed size virtual disk file may take longer to create on some 
p A systems but is often faster to use. 


Storage details 


(+) Dynamically allocated 
O) Fixed size 


Go Back ) | Continue 


Choose between 10GB and 16GB as the disk size. 


Create New Virtual Disk 


Virtual disk file location and size 


Please type the name of the new virtual disk file into the box 
below or click on the folder icon to select a different folder to 
create the file in. 


_ Location 


mijnvirtuelemachine 


d Select the size of the virtual disk in megabytes. This size will be 
reported to the Guest OS as the maximum size of this virtual disk. 


Size 


=r qg 10,00 GB 
4,00 MB 2,00 TB 


H 
( GoBack ) ( Continue ) 
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Click create to create the virtual disk. 


mee Create New Virtual Disk 


Summary 


You are going to create a new virtual disk with the following 
parameters: 


File type: VDI (VirtualBox Disk Image) rj 

Details: Dynamically allocated storage 

Location: /Users/paul/VirtualBox VMs/mijnvirtuelemachine/mi a 
jnvirtuelemachine.vdi + 


A If the above settings are correct, press the Create button. Once 
you press it the new virtual disk file will be created. 


( GoBack ) ( Create ) 


á 


Click create to create the virtual machine. 


ODO Create New Virtual Machine 


Summary 


You are going to create a new virtual machine with the following parameters: 


Name: mijnvirtuelemachine 
OS Type: Ubuntu (64 bit) 
Base Memory: 512 MB 


Start-up Disk: mijnvirtuelemachine.vdi (Normal, 10,00 GB) 

If the above is correct press the Create button. Once you press it, a new virtual 
machine will be created. 

E Note that you can alter these and all other setting of the created virtual machine 
at any time using the Settings dialog accessible through the menu of the main 
window. 


4 
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6.4. attach the CD image 


Before we start the virtual computer, let us take a look at some settings (click Settings). 


eo Oracle VM VirtualBox Manager = 


ER mijnvirtuelemachine [El General 
fq Powered Off 
4 Name:  mijnvirtuelemachine 


€ =~ OS Type: Ubuntu (64 bit) 
System 
y e Base Memory: 512 MB 
Boot Order: Floppy, 
Fr =a CD/DVD-ROM, 
Hard Disk 9 
Acceleration: VT-x/AMD-V, 
Display 
— — 
à | m Video Memory: 12 MB 
Remote Desktop Server: Disabled 
7 go" © Storage 
IDE Controller 
y £e IDE Secondary Master (CD/DVD): Empty 
SATA Controller 
SATA Port 0: mijnvirtuelemachine.vdi 
— o» J 
R = (Normal, 10,00 GB) 


mg | BP Audio 
X yal» 


mijnvirtuelemachine 


air 


Do not worry if your screen looks different, just find the button named storage. 


mijnvirtuelemachine - General 


Name 
Operating System: | Linux B 
f 
Version: | Ubuntu (64 bit) 


© Ccancel ) GOK 
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Remember the .ISO file you downloaded? Connect this .ISO file to this virtual machine by 
clicking on the CD icon next to Empty. 


Storage Tree - | Attributes 
& IDE Controller CD/DVD Drive: | IDE Secondary | $ ] © 
O Empty O Live CD/DVD 
d& SATA Controller E TEUER 
mijnvirtuelemachine.vdi 
Type: -- 
Size: -- 
Location: -- 
Attached To: -- 


CEOE EC 
© Ccancel ) GOK) 


Now click on the other CD icon and attach your ISO file to this virtual CD drive. 


[El Preview 


| mijnvirtuelemachine 
e: Ubuntu (64 bit) 


Storage Tree | Attributes 


[A IDE Controller | CD/DVD Drive: © 


" Choose a virtual CD/DVD disk file... 
, Olive CD/DVD LE & 2 a e o om 
| Information — a 


Type: EI LL... LE 


Size: -- 


| (2 SATA Controller 
mijnvirtuelemachine.vdi 


Location: -- - > +p I L- 
Attached To: 


ondary Master (CD/DVD): Empty 
ontroller 

Port 0: mijnvirtuele 
(Normal, 1C 
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Verify that your download is accepted. If Virtualbox complains at this point, then you 
probably did not finish the download of the CD (try downloading it again). 


mijnvirtuelemachine - Storage 


Bu 8\Q> 2 ® a 


General System Display Audio Network Ports Shared Folders 
Storage Tree Attributes 
& IDE Controller CD/DVD Drive: | IDE Secondary + | © 


| (3) ubuntu-11.04-server-am... 


L]Live CD/DVD 
A SATA Controller Sees 
mijnvirtuelemachine.vdi 
Type: Image 


Size: 673,61 MB 
Location: /Users/paul/ISO/ubu... 
Attached To: -- 


EEES 
© 


It could be useful to set the network adapter to bridge instead of NAT. Bridged usually will 
connect your virtual computer to the Internet. 


mijnvirtuelemachine - Network 


pter Adapter 2  Adapter3 Adapter 4 


MÍ Enable Network Adapter 


Attached to 


<> 


D Advanced 


© Ccancel ) o 
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6.5. install Linux 


The virtual machine is now ready to start. When given a choice at boot, select install and 
follow the instructions on the screen. When the installation is finished, you can log on to 
the machine and start practising Linux! 


68 


Part Ill. first steps on 
the command line 
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Chapter 7. man pages 


This chapter will explain the use of man pages (also called manual pages) on your Unix 
or Linux computer. 


You will learn the man command together with related commands like whereis, whatis 
and mandb. 


Most Unix files and commands have pretty good man pages to explain their use. Man 
pages also come in handy when you are using multiple flavours of Unix or several Linux 
distributions since options and parameters sometimes vary. 
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7.1. man $command 


Type man followed by a command (for which you want help) and start reading. Press q to 
quit the manpage. Some man pages contain examples (near the end). 


paul@laika:~$ man whois 
Reformatting whois(1), please wait... 


7.2. man $configfile 


Most configuration files have their own manual. 


paul@laika:~$ man syslog.conf 
Reformatting syslog.conf(5), please wait... 


7.3. man $daemon 


This is also true for most daemons (background programs) on your system.. 


paul@laika:~$ man syslogd 
Reformatting syslogd(8), please wait... 


7.4. man -k (apropos) 


man -k (or apropos) shows a list of man pages containing a string. 


paul@laika:~$ man -k syslog 


lm-syslog-setup (8) - configure laptop mode to switch syslog.conf 
logger (1) - a shell command interface to the syslog(3) 
syslog-facility (8) - Setup and remove LOCALx facility for sysklogd 
syslog.conf (5) = syslogd(8) configuration file 

syslogd (8) - Linux system logging utilities. 
syslogd-listfiles (8) - list system logfiles 


7.5. whatis 


To see just the description of a manual page, use whatis followed by a string. 


paul@u810:~$ whatis route 
route (8) - show / manipulate the IP routing table 


7.6. whereis 


The location of a manpage can be revealed with whereis. 


paul@laika:~$ whereis -m whois 
whois: /usr/share/man/manl/whois.1.gz 


This file is directly readable by man. 


paul@laika:~$ man /usr/share/man/manl/whois.1.gz 
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7.7. man sections 


By now you will have noticed the numbers between the round brackets. man man will 
explain to you that these are section numbers. Executable programs and shell commands 
reside in section one. 


Executable programs or shell commands 

System calls (functions provided by the kernel) 

Library calls (functions within program libraries) 

Special files (usually found in /dev) 

File formats and conventions eg /etc/passwd 

Games 

Miscellaneous (including macro packages and conventions), e.g. man(7) 
System administration commands (usually only for root) 

Kernel routines [Non standard] 


7.8. man Ssection $file 


wma nub 0) IN H2 


Therefor, when referring to the man page of the passwd command, you will see it written 
as passwd(1); when referring to the passwd file, you will see it written as passwd(5). The 
screenshot explains how to open the man page in the correct section. 


[paul@RHEL52 ~]$ man passwd # opens the first manual found 
[paul@RHEL52 ~]$ man 5 passwd # opens a page from section 5 


7.9. man man 


If you want to know more about man, then Read The Fantastic Manual (RTFM). 


Unfortunately, manual pages do not have the answer to everything... 


paul@laika:~$ man woman 
No manual entry for woman 


7.10. mandb 


Should you be convinced that a man page exists, but you can't access it, then try running 
mandb on Debian/Mint. 


root@laika:~# mandb 

O man subdirectories contained newer manual pages. 
O manual pages were added. 

O stray cats were added. 

O old database entries were purged. 


Or run makewhatis on CentOS/Redhat. 


[root@centos65 ~]# apropos scsi 

scsi: nothing appropriate 

[root@centos65 ~]# makewhatis 

[root@centos65 ~]# apropos scsi 

hpsa (4) - HP Smart Array SCSI driver 

lsscsi (8) - list SCSI devices (or hosts) and their attributes 
sd (4) — Driver for SCSI Disk Drives 

st (4) - SCSI tape device 
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Chapter 8. working with directories 


'This module is a brief overview of the most common commands to work with directories: 
pwd, cd, ls, mkdir and rmdir. These commands are available on any Linux (or Unix) 
system. 


This module also discusses absolute and relative paths and path completion in the bash 
shell. 
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8.1. pwd 


The you are here sign can be displayed with the pwd command (Print Working Directory). 
Go ahead, try it: Open a command line interface (also called a terminal, console or xterm) 
and type pwd. The tool displays your current directory. 


paulGdebian8:-$ pwd 
/home/paul 


8.2. cd 


You can change your current directory with the cd command (Change Directory). 


paulGdebian8$ cd /etc 
paul@debian8$ pwd 
/etc 
paul@debian8$ cd /bin 
paulGdebian8$ pwd 
/bin 
paul@debian8$ cd /home/paul/ 
paul@debian8$ pwd 

/home/paul 


8.2.1. cd ~ 


The cd is also a shortcut to get back into your home directory. Just typing ed without a target 
directory, will put you in your home directory. Typing cd ~ has the same effect. 


paul@debian8$ cd /etc 
paul@debian8$ pwd 
/etc 
paul@debian8$ cd 
paulGdebian8$ pwd 
/home/paul 
paul@debian8$ cd ~ 
paul@debian8$ pwd 
/home/paul 


8.2.2. cd .. 


To go to the parent directory (the one just above your current directory in the directory 
tree), type cd... 


paul@debian8$ pwd 
/usr/share/games 
paul@debian8$ cd .. 
paul@debian8$ pwd 
/usr/share 


To stay in the current directory, type cd . ;-) We will see useful use of the . character 
representing the current directory later. 
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8.2.3. cd - 


Another useful shortcut with cd is to just type cd - to go to the previous directory. 


paul@debian8$ 
/home/paul 

paul@debian8$ 
paul@debian8$ 
/etc 
paul@debian8$ 


pwd 


cd /etc 
pwd 


cd - 


/home/paul 
paul@debian8$ 
/etc 


8.3. absolute and relative paths 


cd - 


You should be aware of absolute and relative paths in the file tree. When you type a path 
starting with a slash (/), then the root of the file tree is assumed. If you don't start your path 
with a slash, then the current directory is the assumed starting point. 


The screenshot below first shows the current directory /home/paul. From within this 
directory, you have to type cd /home instead of cd home to go to the /home directory. 


paul@debian8$ pwd 


/home/paul 
paul@debian8$ cd home 
bash: cd: home: No such file or directory 


paul@debian8$ cd /home 
paul@debian8$ pwd 
/home 


When inside /home, you have to type cd paul instead of cd /paul to enter the subdirectory 
paul of the current directory /home. 


paul@debian8$ pwd 

/home 

paul@debian8$ cd /paul 

bash: cd: /paul: No such file or directory 
paul@debian8$ cd paul 

paulGdebian8$ pwd 

/home/paul 


In case your current directory is the root directory /, then both cd /home and cd home will 
get you in the /home directory. 


paul@debian8$ 
// 


pwd 


paul@debian8$ 
paul@debian8$ 
/home 

paul@debian8$ 
paul@debian8$ 
paul@debian8$ 
/home 


cd home 
pwd 


cd / 
cd /home 
pwd 


This was the last screenshot with pwd statements. From now on, the current directory will 
often be displayed in the prompt. Later in this book we will explain how the shell variable 
$PS1 can be configured to show this. 


76 


working with directories 


8.4. path completion 


The tab key can help you in typing a path without errors. Typing cd /et followed by the tab 
key will expand the command line to cd /etc/. When typing cd /Et followed by the tab key, 
nothing will happen because you typed the wrong path (upper case E). 


You will need fewer key strokes when using the tab key, and you will be sure your typed 
path is correct! 


8.5. Is 


You can list the contents of a directory with Is. 


paul@debian8:~$ ls 
allfiles.txt  dmesg.txt services stuff  summer.txt 
paul@debian8:~$ 


8.5.1. Is -a 


A frequently used option with Is is -a to show all files. Showing all files means including 
the hidden files. When a file name on a Linux file system starts with a dot, it is considered 
a hidden file and it doesn't show up in regular file listings. 


paul@debian8:~$ ls 
allfiles.txt dmesg.txt services stuff summer.txt 
paul@debian8:~$ ls -a 
allfiles.txt .bash profile  dmesg.txt .lesshst stuff 
.bash history  .bashrc services «ssh summer .t xt 
paul@debian8:~$ 


8.5.2. Is -l 


Many times you will be using options with Is to display the contents of the directory in 
different formats or to display different parts of the directory. Typing just Is gives you a 
list of files in the directory. Typing Is -l (that is a letter L, not the number 1) gives you a 
long listing. 

paul@debian8:~$ ls -1 


tota ly 2396 
-rw-r--r-- 1 paul paul 17584442 Sep 17 00:03 allfiles.txt 


aye cae d joys Mi paul 96650 Sep 17 00:03 dmesg.txt 
srw e e TL OEM lipan 19558 Sep 17 00:04 services 
drwxr-xr-x 2 paul paul 4096 Sep 17 00:04 stuff 
AN I paul paul 0 Sep 17 00:04 summer.txt 
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8.5.3. Is -Ih 


Another frequently used Is option is -h. It shows the numbers (file sizes) in a more human 
readable format. Also shown below is some variation in the way you can give the options 
to Is. We will explain the details of the output later in this book. 


Note that we use the letter L as an option in this screenshot, not the number 1. 


paul@debian8:~$ ls -l -h 


total 17M 

=rw- r- r= | paul paul 17M Sep 17 00703 allfiles txt 
-rw-r--r-- 1 paul paul 95K Sep 17 00:03 dmesg.txt 
-rw-r--r-- 1 paul paul 20K Sep 17 00:04 services 
drwxr-xr-x 2 paul paul 4.0K Sep 17 00:04 stuff 
crew E E l paul paur 0 Sep 17 00:04 summer.txt 
paulGdebian8:-$ ls -1h 

total 17M 

Suv ae l paul paul 17M Sep 17 10/08 OS Ra Mires Ese 
-rw-r--r-- 1 paul paul 95K Sep 17 00:03 dmesg.txt 
-rw-r--r-- 1 paul paul 20K Sep 17 00:04 services 
drwxr-xr-x 2 paul paul 4.0K Sep 17 00:04 stuff 

PENES ee paul aus 0 Sep 17 00:04 summer.txt 
paul@debian8:~$ ls -hl 

total 17M 

SW r- r l paul paul M Sep) 17 004095 all tales txt 
crwt- r= 1 paul paul 95K Sep 17 00:039 amesg. txe 
-rw-r--r-- 1 paul paul 20K Sep 17 00:04 services 
drwxr-xr-x 2 paul paul 4.0K Sep 17 00:04 stuff 
-rw r- re TE paul paul 0 Sep 17 00:04 summer.txt 
paul@debian8:~$ ls -h -1 

total 17M 

=rw-r==-r== 1 paul paul 17M Sep 17 00:03 allfiles, txt 
=rw r r l paul paul 95K Sep 17 00:09 dmesg .txt 
-rw-r--r-- 1 paul paul 20K Sep 17 00:04 services 
drwxr-xr-x 2 paul paul 4.0K Sep 17 00:04 stuff 
SS C l paul paul 0 Sep 17 00:04 summer.txt 


paul@debian8:~$ 
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8.6. mkdir 


Walking around the Unix file tree is fun, but itis even more fun to create your own directories 
with mkdir. You have to give at least one parameter to mkdir, the name of the new directory 
to be created. Think before you type a leading / . 


paul@debian8:~$ mkdir mydir 

paul@debian8:~$ cd mydir 

paul@debian8:~/mydir$S ls -al 

total 8 

drwxr-xr-x 2 paul paul 4096 Sep 17 00:07 
drwxr-xr-x 48 paul paul 4096 Sep 17 00:07 
paul@debian8:~/mydirS mkdir stuff 
paulGdebian8:-/mydir$ mkdir otherstuff 
paul@debian8:~/mydir$ ls -1 

total 8 

drwxr-xr-x 2 paul paul 4096 Sep 17 00:08 otherstuff 
drwxr-xr-x 2 paul paul 4096 Sep 17 00:08 stuff 
paul@debian8:~/mydir$ 


8.6.1. mkdir -p 


The following command will fail, because the parent directory of threedirsdeep does not 
exist. 


paul@debian8:~$ mkdir mydir2/mysubdir2/threedirsdeep 
mkdir: cannot create directory ‘mydir2/mysubdir2/threedirsdeep’: No such fi\ 
le or directory 


When given the option -p, then mkdir will create parent directories as needed. 


paul@debian8:~$ mkdir -p mydir2/mysubdir2/threedirsdeep 
paul@debian8:~$ cd mydir2 

paul@debian8:~/mydir2$ 1s -1 

total 4 

drwxr-xr-x 3 paul paul 4096 Sep 17 00:11 mysubdir2 
paul@debian8:~/mydir2S cd mysubdir2 
paul@debian8:~/mydir2/mysubdir2$ 1s -1 

total 4 

drwxr-xr-x 2 paul paul 4096 Sep 17 00:11 threedirsdeep 
paul@debian8:~/mydir2/mysubdir2$ cd threedirsdeep/ 
paul@debian8:~/mydir2/mysubdir2/threedirsdeepS pwd 
/home/paul/mydir2/mysubdir2/threedirsdeep 


8.7. rmdir 


When a directory is empty, you can use rmdir to remove the directory. 


paul@debian8:~/mydir$ ls -1 

total 8 

drwxr-xr-x 2 paul paul 4096 Sep 17 00:08 otherstuff 
drwxr-xr-x 2 paul paul 4096 Sep 17 00:08 stuff 
paul@debian8:~/mydir$ rmdir otherstuff 
paul@debian8:~/mydir$ ed .. 

paul@debian8:~$ rmdir mydir 

rmdir: failed to remove ‘mydir’: Directory not empty 
paul@debian8:~$ rmdir mydir/stuff 

paul@debian8:~$ rmdir mydir 

paul@debian8:~$ 
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8.7.1. rmdir -p 
And similar to the mkdir -p option, you can also use rmdir to recursively remove 
directories. 
paul@debian8:~$ mkdir -p test42/subdir 


paul@debian8:~$ rmdir -p test42/subdir 
paulGdebian8:-$ 
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8.8. practice: working with directories 


1. Display your current directory. 

2. Change to the /etc directory. 

3. Now change to your home directory using only three key presses. 
4. Change to the /boot/grub directory using only eleven key presses. 
5. Go to the parent directory of the current directory. 

6. Go to the root directory. 

7. List the contents of the root directory. 

8. List a long listing of the root directory. 

9. Stay where you are, and list the contents of /etc. 

10. Stay where you are, and list the contents of /bin and /sbin. 

11. Stay where you are, and list the contents of ~. 

12. List all the files (including hidden files) in your home directory. 
13. List the files in /boot in a human readable format. 

14. Create a directory testdir in your home directory. 


15. Change to the /etc directory, stay here and create a directory newdir in your home 
directory. 


16. Create in one command the directories ~/dir1/dir2/dir3 (dir3 is a subdirectory from dir2, 
and dir2 is a subdirectory from dirl ). 


17. Remove the directory testdir. 
18. If time permits (or if you are waiting for other students to finish this practice), use and 


understand pushd and popd. Use the man page of bash to find information about these 
commands. 


81 


working with directories 


8.9. solution: working with directories 


1. Display your current directory. 

pwd 

2. Change to the /etc directory. 

camere 

3. Now change to your home directory using only three key presses. 
cd (and the enter key) 

4. Change to the /boot/grub directory using only eleven key presses. 
cd /boot/grub (use the tab key) 

5. Go to the parent directory of the current directory. 

cd .. (with space between cd and ..) 

6. Go to the root directory. 

ecl y 

7. List the contents of the root directory. 

iis 

8. List a long listing of the root directory. 

ls -1 

9. Stay where you are, and list the contents of /etc. 

sy /cue 

10. Stay where you are, and list the contents of /bin and /sbin. 

ls /bin /sbin 

11. Stay where you are, and list the contents of ~. 

ls ~ 

12. List all the files (including hidden files) in your home directory. 
ls -al ~ 

13. List the files in /boot in a human readable format. 

ls -lh /boot 

14. Create a directory testdir in your home directory. 


mkdir ~/testdir 


15. Change to the /etc directory, stay here and create a directory newdir in your home 
directory. 
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cd /etc ; mkdir ~/newdir 


16. Create in one command the directories ~/dir1/dir2/dir3 (dir3 is a subdirectory from dir2, 
and dir2 is a subdirectory from dirl ). 


mkdir -p -/diri/dir2/dir3 


17. Remove the directory testdir. 


rmdir testdir 


18. If time permits (or if you are waiting for other students to finish this practice), use and 
understand pushd and popd. Use the man page of bash to find information about these 
commands. 


man bash # opens the manual 
/pushd # searches for pushd 
n # next (do this two/three times) 


The Bash shell has two built-in commands called pushd and popd. Both commands work 
with a common stack of previous directories. Pushd adds a directory to the stack and changes 
to a new current directory, popd removes a directory from the stack and sets the current 
directory. 


paul@debian7:/etcS cd /bin 
paulGdebian7:/bin$ pushd /lib 
E813 55 Aia 

paul8debian7:/1ib$ pushd /proc 
hase Allo /pin 
paul@debian7:/proc$ popd 

(is: Ain 

paul@debian7:/libS popd 

/bin 
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Chapter 9. working with files 


In this chapter we learn how to recognise, create, remove, copy and move files using 
commands like file, touch, rm, cp, mv and rename. 
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9.1. all files are case sensitive 


Files on Linux (or any Unix) are case sensitive. This means that FILE1 is different from 
filel, and /etc/hosts is different from /etc/Hosts (the latter one does not exist on a typical 
Linux computer). 


This screenshot shows the difference between two files, one with upper case W, the other 
with lower case w. 


paul@laika:~/Linux$ 1s 

winter.txt Winter.txt 
paul@laika:~/Linux$ cat winter.txt 
TE TS Cold, 
paul@laika:~/Linux$ cat Winter.txt 
Te is very cold! 


9.2. everything is a file 


A directory is a special kind of file, but it is still a (case sensitive!) file. Each terminal 
window (for example /dev/pts/4), any hard disk or partition (for example /dev/sdb1) and 
any process are all represented somewhere in the file system as a file. It will become clear 
throughout this course that everything on Linux is a file. 


9.3. file 


The file utility determines the file type. Linux does not use extensions to determine the 
file type. The command line does not care whether a file ends in .txt or .pdf. As a system 
administrator, you should use the file command to determine the file type. Here are some 
examples on a typical Linux system. 


paul@laika:~$ file pic33.png 

pic33.png: PNG image data, 3840 x 1200, 8-bit/color RGBA, non-interlaced 
paul@laika:~$ file /etc/passwd 

/etc/passwd: ASCII text 

paul@laika:~$ file HelloWorld.c 

HelloWorld.c: ASCII C program text 


The file command uses a magic file that contains patterns to recognise file types. The magic 
file is located in /usr/share/file/magic. Type man 5 magic for more information. 


It is interesting to point out file -s for special files like those in /dev and /proc. 


root@debian6~# file /dev/sda 

/dev/sda: block special 

root@debian6~# file -s /dev/sda 

/dev/sda: x86 boot sector; partition 1: ID-0x83, active, starthead... 
root@debian6~# file /proc/cpuinfo 

/proc/cpuinfo: empty 

root@debian6~# file -s /proc/cpuinfo 

/proc/cpuinfo: ASCII C++ program text 


85 


working with files 


9.4. touch 


9.4.1. create an empty file 


One easy way to create an empty file is with touch. (We will see many other ways for 


creating files later in this book.) 


This screenshot starts with an empty directory, creates two files with touch and the lists 


those files. 


paul@debian7:~$ ls -1 
total 0 


paul@debian7:~$ touch file42 
paulGdebian7:-$ touch file33 


paul@debian7:~$ ls -1 
total 0 


aye r l paul paul (0) oret 
-rwr r TE paul paul 0) erg 


paul@debian7:~$ 


9.4.2. touch -t 


RSMO Soe) cales 
T5 08:756 file42 


The touch command can set some properties while creating empty files. Can you determine 
what is set by looking at the next screenshot? If not, check the manual for touch. 


paul@debian7:~$ touch -t 
paul@debian7:~$ touch -t 


paul@debian7:~$ ls -1 
total 0 


Wai 
[ley cae uec I paul paul 
[impie r paul paul 
mrw- e- e paul paul 


paul@debian7:~$ 


O O OG 


200505050000 SinkoDeMayo 
130207111630 


Jul 
Oct 
OcE 
May 


r 
15 
L5 

5 


1302 
(QOIS DU) 
OST 1D 6 

2005 


BigBattle.txt 


BigBattle.txt 
file33 

file42 
SinkoDeMayo 
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9.5. rm 


9.5.1. remove forever 


When you no longer need a file, use rm to remove it. Unlike some graphical user interfaces, 
the command line in general does not have a waste bin or trash can to recover files. When 
you use rm to remove a file, the file is gone. Therefore, be careful when removing files! 


paul@debian7:~$ ls 

BigBattle.txt  file33 file42 SinkoDeMayo 
paul@debian7:~$ rm BigBattle.txt 
paul@debian7:~$ 1s 

file33 file42 SinkoDeMayo 
paul@debian7:~S$ 


9.5.2. rm -i 


To prevent yourself from accidentally removing a file, you can type rm -i. 


paul@debian7:~$ 1s 

file33 file42 SinkoDeMayo 

paul@debian7:~$ rm -i file33 

rm: remove regular empty file ^file33'? yes 
paul@debian7:~S rm -i SinkoDeMayo 

rm: remove regular empty file ~SinkoDeMayo'? n 
paul@debian7:~$ ls 

file42  SinkoDeMayo 

paul@debian7:~$ 


9.5.3. rm -rf 


By default, rm -r will not remove non-empty directories. However rm accepts several 
options that will allow you to remove any directory. The rm -rf statement is famous because 
it will erase anything (providing that you have the permissions to do so). When you are 
logged on as root, be very careful with rm -rf (the f means force and the r means recursive) 
since being root implies that permissions don't apply to you. You can literally erase your 
entire file system by accident. 


paul@debian7:~$ mkdir test 

paul@debian7:~$ rm test 

rm: cannot remove 'test': Is a directory 
paul@debian7:~$ rm -rf test 

paul@debian7:~$ ls test 

ls: cannot access test: No such file or directory 
paul@debian7:~$ 
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9.6. cp 
9.6.1. copy one file 


To copy a file, use cp with a source and a target argument. 


paul@debian7:~$ ls 

file42  SinkoDeMayo 

paul@debian7:~$ cp file42 file42.copy 
paul@debian7:~$ 1s 

file42 file42.copy SinkoDeMayo 


9.6.2. copy to another directory 


If the target is a directory, then the source files are copied to that target directory. 


paulGdebian7:-$ mkdir dir42 
paul@debian7:~$ cp SinkoDeMayo dir42 
paul@debian7:~$ ls dir42/ 
SinkoDeMayo 


9.6.3. cp -r 


To copy complete directories, use cp -r (the -r option forces recursive copying of all files 
in all subdirectories). 


paul@debian7:~$ ls 

dir42 file42 file42.copy SinkoDeMayo 
paul@debian7:~$ cp -r dir42/ dir33 
paul@debian7:~$ ls 

dir33 dir42 file42 file42.copy SinkoDeMayo 
paul@debian7:~$ ls dir33/ 

SinkoDeMayo 


9.6.4. copy multiple files to directory 


You can also use cp to copy multiple files into a directory. In this case, the last argument 
(a.k.a. the target) must be a directory. 


paul@debian7:~$ cp file42 file42.copy SinkoDeMayo dir42/ 
paul@debian7:~$ ls dir42/ 
file42 file42.copy SinkoDeMayo 


9.6.5. cp -i 


To prevent cp from overwriting existing files, use the -i (for interactive) option. 


paul@debian7:~$ cp SinkoDeMayo file42 
paul@debian7:~$ cp SinkoDeMayo file42 
paulGdebian7:-$ cp -i SinkoDeMayo file42 
cp: overwrite ^file42'? n 
paulGdebian7:-$ 
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9.7. mv 


9.7.1. rename files with mv 


Use mv to rename a file or to move the file to another directory. 


paul@debian7:~$ ls 

dir33 dir42 file42 file42.copy SinkoDeMayo 
paul@debian7:~$ mv file42 file33 
paul@debian7:~$ ls 

dir33 dir42 file33 file42.copy SinkoDeMayo 
paul@debian7:~$ 


When you need to rename only one file then mv is the preferred command to use. 


9.7.2. rename directories with mv 


The same mv command can be used to rename directories. 


paul@debian7:~$ ls -1 
total 8 


drwxr=-xr:=x 2 paul paul 4096 10ct T5 109; Soma 3S 
drwxr-xr-x 2 paul paul 4096 Oct 15 09:36 dir42 
SEW b= l paul paul OFOCE 15 09:38 files 
zew- r- TE joe yell paul 0 Oct 15 09:16 file42.copy 
SE ro A paul paul 0 May 5 2005 SinkoDeMayo 


paul@debian7:~$ mv dir33 backup 
paul@debian7:~$ ls -1 
gota m d 


drwxr-xr-x 2 paul paul 4096 Oct 15 09:36 backup 
drwxr-xr-x 2 paul paul 4096 Oct 15 09:36 dir42 

aye nau paul ONO CE 15 1109:38 files 
crew e > paul paul o oct 15 09:16 file42. copy: 
mrw e- e dE paullpaul 0 May 5 2005 SinkoDeMayo 


paul@debian7:~$ 
9.7.3. mv -i 


The mv also has a -i switch similar to cp and rm. 


this screenshot shows that mv -i will ask permission to overwrite an existing file. 
paulGdebian7:-$ mv -i file33 SinkoDeMayo 


mv: overwrite ^SinkoDeMayo'? no 
paulGdebian7:-$ 
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9.8. rename 


9.8.1. about rename 


The rename command is one of the rare occasions where the Linux Fundamentals book 
has to make a distinction between Linux distributions. Almost every command in the 
Fundamentals part of this book works on almost every Linux computer. But rename is 
different. 


Try to use mv whenever you need to rename only a couple of files. 


9.8.2. rename on Debian/Ubuntu 


The rename command on Debian uses regular expressions (regular expression or shor regex 
are explained in a later chapter) to rename many files at once. 


Below a rename example that switches all occurrences of txt to png for all file names ending 
in .txt. 


paul@debian7:~/test42$S ls 

abc.txt  file33.txt  file42.txt 
paul@debian7:~/test42S rename 's/\.txt/\.png/' *.txt 
paulGdebian7:-/test42$ ls 

abc.png file33.png file42.png 


This second example switches all (first) occurrences of file into document for all file names 
ending in .png. 


paulGdebian7:-/test42$ ls 

abc.png file33.png file42.png 

paul@debian7:~/test42$ rename 's/file/document/' *.png 
paulGdebian7:-/test42$ ls 

abc.png document33.png document42.png 
paul@debian7:~/test42$ 


9.8.3. rename on CentOS/RHEL/Fedora 


On Red Hat Enterprise Linux, the syntax of rename is a bit different. The first example 
below renames all *.conf files replacing any occurrence of .conf with .backup. 


[paul@centos7 ~]$ touch one.conf two.conf three.conf 
[paul@centos7 ~]$ rename .conf .backup *.conf 
[paul@centos7 ~]$ 1s 

one.backup three.backup two.backup 

[paul@centos7 -]$ 


The second example renames all (*) files replacing one with ONE. 


[paul@centos7 ~]$ ls 

one.backup three.backup two.backup 
[paul@centos7 ~]$ rename one ONE * 
[paul@centos7 ~]$ 1s 

ONE.backup three.backup two.backup 
[paul@centos7 ~]$ 
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9.9. practice: working with files 


1. List the files in the /bin directory 
2. Display the type of file of /bin/cat, /etc/passwd and /usr/bin/passwd. 


3a. Download wolf.jpg and LinuxFun.pdf from http://linux-training.be (wget http:// 
linux-training.be/files/studentfiles/wolf.jpg and wget http://linux-training.be/files/books/ 
LinuxFun.pdf) 


wget http://linux-training.be/files/studentfiles/wolf.jpg 
wget http://linux-training.be/files/studentfiles/wolf.png 
wget http://linux-training.be/files/books/LinuxFun.pdf 


3b. Display the type of file of wolf.jpg and LinuxFun.pdf 

3c. Rename wolf.jpg to wolf.pdf (use mv). 

3d. Display the type of file of wolf.pdf and LinuxFun.pdf. 

4. Create a directory ~/touched and enter it. 

5. Create the files today.txt and yesterday.txt in touched. 

6. Change the date on yesterday.txt to match yesterday's date. 

7. Copy yesterday.txt to copy.yesterday.txt 

8. Rename copy.yesterday.txt to kim 

9. Create a directory called ~/testbackup and copy all files from ~/touched into it. 
10. Use one command to remove the directory ~/testbackup and all files into it. 


11. Create a directory ~/etcbackup and copy all *.conf files from /etc into it. Did you include 
all subdirectories of /etc ? 


12. Use rename to rename all *.conf files to *.backup . (if you have more than one distro 
available, try it on all!) 
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9.10. solution: working with files 


1. List the files in the /bin directory 


ls /bin 


2. Display the type of file of /bin/cat, /etc/passwd and /usr/bin/passwd. 


file /bin/cat /etc/passwd /usr/bin/passwd 


3a. Download wolf.jpg and LinuxFun.pdf from http://linux-training.be (wget http:// 
linux-training.be/files/studentfiles/wolf.jpg and wget http://linux-training.be/files/books/ 
LinuxFun.pdf) 


wget http://linux-training.be/files/studentfiles/wolf.jpg 
wget http://linux-training.be/files/studentfiles/wolf.png 
wget http://linux-training.be/files/books/LinuxFun.pdf 


3b. Display the type of file of wolf.jpg and LinuxFun.pdf 
file wolf.jpg LinuxFun.pdf 

3c. Rename wolf.jpg to wolf.pdf (use mv). 

mv wolf.jpg wolf.pdf 

3d. Display the type of file of wolf.pdf and LinuxFun.pdf. 
file wolf.pdf LinuxFun.pdf 

4. Create a directory ~/touched and enter it. 

mkdir ~/touched ; cd ~/touched 

5. Create the files today.txt and yesterday.txt in touched. 


touch today.txt yesterday.txt 


6. Change the date on yesterday.txt to match yesterday's date. 


touch -t 200810251405 yesterday.txt (substitute 20081025 with yesterday) 
7. Copy yesterday.txt to copy.yesterday.txt 

cp yesterday.txt copy.yesterday.txt 

8. Rename copy.yesterday.txt to kim 

mv copy.yesterday.txt kim 

9. Create a directory called ~/testbackup and copy all files from ~/touched into it. 
mkdir -/testbackup ; cp -r -/touched -/testbackup/ 

10. Use one command to remove the directory ~/testbackup and all files into it. 


rm -rf ~/testbackup 


11. Create a directory ~/etcbackup and copy all *.conf files from /etc into it. Did you include 
all subdirectories of /etc ? 
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cp -r /etc/*.conf -/etcbackup 


Only =. conf files that are directly an /etc/ are copied. 


12. Use rename to rename all *.conf files to *.backup . (if you have more than one distro 
available, try it on all!) 


On RHEL: touch 1.conf 2.conf ; rename conf backup *.conf 


On Debian: touch 1.conf 2.conf ; rename 's/conf/backup/' *.conf 
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Chapter 10. working with file contents 


In this chapter we will look at the contents of text files with head, tail, cat, tac, more, less 
and strings. 


We will also get a glimpse of the possibilities of tools like cat on the command line. 
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10.1. head 


You can use head to display the first ten lines of a file. 


paulGdebian7-$ head /etc/passwd 

root Ix: 00 moot / root: bin/basin 
daemon:x:1:1:daemon:/usr/sbin:/bin/sh 
Pin vba vb says 
sys:x:3:3:sys:/dev:/bin/sh 

Ssynerx 741655384 syne/bins/bami/isynic 
games:x:5:60:games:/usr/games:/bin/sh 
man:x:6:12:man:/var/cache/man:/bin/sh 
lp:x: ae Spool, Ipe bin/sh 
malls 8 qmail: vata) mani Dans 
news:x:9:9:news:/var/spool/news:/bin/sh 
rootfdebian7-+ 


The head command can also display the first n lines of a file. 
paul@debian7~S head -4 /etc/passwd 
BOJA uo Ole / root: / bin/bash 
daemon:x:1:1:daemon:/usr/sbin:/bin/sh 

pinez: 272: bin: / pin: ATIS em ESI 

Sisi SS si Geway BIN/SH 

paul@debian7~$ 


And head can also display the first n bytes. 


paul@debian7~$ head -c14 /etc/passwd 
root:x:0:0:roopaul@debian7~$ 


10.2. tail 


Similar to head, the tail command will display the last ten lines of a file. 


paul@debian7~S tail /etc/services 


vboxd 20012/udp 

binkp 24554/tcp # binkp fidonet protocol 

asp 234 tco 4 Address Search Protocol 

asp 27374/udp 

csync2 30865/tcp # cluster synchronization tool 
dircproxy 57000/tcp # Detachable IRC Proxy 

EErdO 60177/tcp + fidonet EMSI over telnet 
fido 60179/tcp # fidonet EMSI over TCP 


# Local services 
paul@debian7~$ 


You can give tail the number of lines you want to see. 


paulGdebian7-$ tail -3 /etc/services 
fido (910 TS AB e19) 4 fidonet EMSI over TCP 


# Local services 
paul@debian7~$ 


The tail command has other useful options, some of which we will use during this course. 
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10.3. cat 


The cat command is one of the most universal tools, yet all it does is copy standard input to 
standard output. In combination with the shell this can be very powerful and diverse. Some 
examples will give a glimpse into the possibilities. The first example is simple, you can use 
cat to display a file on the screen. If the file is longer than the screen, it will scroll to the end. 


paul@debian8:~$ cat /etc/resolv.conf 
domain linux-training.be 

search linux-training.be 

nameserver 192.168.1.42 


10.3.1. concatenate 


cat is short for concatenate. One of the basic uses of cat is to concatenate files into a bigger 
(or complete) file. 


paul@debian8:~$ echo one >partl 
paul@debian8:~$ echo two >part2 
paul@debian8:~$ echo three >part3 
paul@debian8:~$ cat partl 

one 
paul@debian8:~$ cat part2 
two 
paul@debian8:~$ cat part3 

three 

paul@debian8:~$ cat partl part2 part3 

one 

two 

three 

paul@debian8:~$ cat partl part2 part3 >all 
paul@debian8:~$ cat all 

one 

two 

three 

paul@debian8:~$ 


10.3.2. create files 


You can use cat to create flat text files. Type the cat > winter.txt command as shown in the 
screenshot below. Then type one or more lines, finishing each line with the enter key. After 
the last line, type and hold the Control (Ctrl) key and press d. 


paul@debian8:~$ cat > winter.txt 
It is very cold today! 
paul@debian8:~$ cat winter.txt 
It is very cold today! 
paul@debian8:~$ 


The Ctrl d key combination will send an EOF (End of File) to the running process ending 
the cat command. 
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10.3.3. custom end marker 


You can choose an end marker for cat with << as is shown in this screenshot. This 
construction is called a here directive and will end the cat command. 


paul@debian8:~$ cat > hot.txt <<stop 
> It is hot today! 

> Yes it is summer. 

= SLOp 

paul@debian8:~$ cat hot.txt 

It is hot today! 

Yes it is summer. 

paul@debian8:~$ 


10.3.4. copy files 


In the third example you will see that cat can be used to copy files. We will explain in detail 
what happens here in the bash shell chapter. 


paul@debian8:~$ cat winter.txt 

It is very cold today! 

paul@debian8:~$ cat winter.txt > cold.txt 
paul@debian8:~$ cat cold.txt 

It is very cold today! 

paulGdebian8:-$ 


10.4. tac 


Just one example will show you the purpose of tac (cat backwards). 


paul@debian8:~$ cat count 
one 

two 

three 

four 

paulGdebian8:-$ tac count 
BE CODE 

three 

two 

one 
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10.5. more and less 


The more command is useful for displaying files that take up more than one screen. More 
will allow you to see the contents of the file page by page. Use the space bar to see the next 
page, or q to quit. Some people prefer the less command to more. 


10.6. strings 


With the strings command you can display readable ascii strings found in (binary) files. 
This example locates the Is binary then displays readable strings in the binary file (output 
is truncated). 


paul@laika:~$ which ls 
/bin/ls 
paul@laika:~$ strings /bin/ls 
apran iinux: SoZ 
librt.so.l 

gmon_start 
_Jv_RegisterClasses 
clock_gettime 
libacl . soi. 
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10.7. practice: file contents 


1. Display the first 12 lines of /etc/services. 
2. Display the last line of /etc/passwd. 


3. Use cat to create a file named count.txt that looks like this: 


One 
Two 
Three 
Four 
Five 


4. Use cp to make a backup of this file to cnt.txt. 

5. Use cat to make a backup of this file to catent.txt. 

6. Display catent.txt, but with all lines in reverse order (the last line first). 

7. Use more to display /etc/services. 

8. Display the readable character strings from the /usr/bin/passwd command. 
9. Use Is to find the biggest file in /etc. 


10. Open two terminal windows (or tabs) and make sure you are in the same directory in 
both. Type echo this is the first line > tailing.txt in the first terminal, then issue tail -f 
tailing.txt in the second terminal. Now go back to the first terminal and type echo This is 
another line >> tailing.txt (note the double >>), verify that the tail -f in the second terminal 
shows both lines. Stop the tail -f with Ctrl-C. 


11. Use catto create a file named tailing.txt that contains the contents of tailing.txt followed 
by the contents of /etc/passwd. 


12. Usecatto create a file named tailing.txt that contains the contents of tailing.txt preceded 
by the contents of /etc/passwd. 
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10.8. solution: file contents 


1. Display the first 12 lines of /etc/services. 

head -12 /etc/services 

2. Display the last line of /etc/passwd. 

tail -1 /etc/passwd 

3. Use cat to create a file named count.txt that looks like this: 
Gat > count .txt 

One 

Two 

Three 


Four 
Five (followed by Ctrl-d) 


4. Use cp to make a backup of this file to cnt.txt. 


Cp Count CXE CNE. CXE 


5. Use cat to make a backup of this file to catent.txt. 


cat count.txt > catent.txt 


6. Display catent.txt, but with all lines in reverse order (the last line first). 
tac catent.txt 

7. Use more to display /etc/services. 

more /etc/services 

8. Display the readable character strings from the /usr/bin/passwd command. 
strings /usr/bin/passwd 

9. Use Is to find the biggest file in /etc. 


ls -lrS /etc 


10. Open two terminal windows (or tabs) and make sure you are in the same directory in 
both. Type echo this is the first line > tailing.txt in the first terminal, then issue tail -f 
tailing.txt in the second terminal. Now go back to the first terminal and type echo This is 
another line >> tailing.txt (note the double >>), verify that the tail -f in the second terminal 
shows both lines. Stop the tail -f with Ctrl-C. 


11. Use cat to create a file named tailing.txt that contains the contents of tailing.txt followed 
by the contents of /etc/passwd. 


cat /etc/passwd >> tailing.txt 


12. Use cat to create a file named tailing.txt that contains the contents of tailing.txt preceded 
by the contents of /etc/passwd. 


mv tailing.txt tmp.txt ; cat /etc/passwd tmp.txt » tailing.txt 
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This chapter takes a look at the most common directories in the Linux file tree. It also shows 
that on Unix everything is a file. 
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11.1. filesystem hierarchy standard 


Many Linux distributions partially follow the Filesystem Hierarchy Standard. The FHS 
may help make more Unix/Linux file system trees conform better in the future. The FHS 
is available online at http://www.pathname.com/fhs/ where we read: "The filesystem 
hierarchy standard has been designed to be used by Unix distribution developers, package 
developers, and system implementers. However, it is primarily intended to be a reference 
and is not a tutorial on how to manage a Unix filesystem or directory hierarchy." 


11.2. man hier 


There are some differences in the filesystems between Linux distributions. For help about 
your machine, enter man hier to find information about the file system hierarchy. This 
manual will explain the directory structure on your computer. 


11.3. the root directory / 


All Linux systems have a directory structure that starts at the root directory. The root 
directory is represented by a forward slash, like this: /. Everything that exists on your Linux 
system can be found below this root directory. Let's take a brief look at the contents of the 
root directory. 


[paul@RHELv4u3 -]$ ls / 
bin dev home media mnt proc sbin Srv aaa USE 
boot etc lib misc opt root selinux sys tmp var 
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11.4. binary directories 


Binaries are files that contain compiled source code (or machine code). Binaries can be 
executed on the computer. Sometimes binaries are called executables. 


11.4.1. /bin 


The /bin directory contains binaries for use by all users. According to the FHS the /bin 
directory should contain /bin/cat and /bin/date (among others). 


In the screenshot below you see common Unix/Linux commands like cat, cp, cpio, date, dd, 


echo, grep, and so on. Many of these will be covered in this book. 


paul@laika:~$ ls /bin 


archdetect egrep mt setupcon 
autopartition false mt-gnu sh 

bash fgconsole mv Si datsini» 
bunzip2 fgrep nano Sleep 
bzcat fuser nc stralign 
bzcmp fusermount nc.traditional stty 
pza get mountoptions  netcat su 
bzegrep grep netstat sync 
bzexe gunzip DetS S9 sysfs 
bzfgrep gzexe ntfs-3g.probe tailf 
bzgrep (ofa atje) parted_devices tar 
bzip2 hostname parted_server tempfile 
bzip2recover hw-detect partman touch 
bzless al partman-commit true 
bzmore kbd mode perform recipe ulockmgr 
cat kari pidof umount 


11.4.2. other /bin directories 


You can find a /bin subdirectory in many other directories. A user named serena could put 
her own programs in /home/serena/bin. 


Some applications, often when installed directly from source will put themselves in /opt. A 
samba server installation can use /opt/samba/bin to store its binaries. 


11.4.3. /sbin 


/sbin contains binaries to configure the operating system. Many of the system binaries 
require root privilege to perform certain tasks. 


Below a screenshot containing system binaries to change the ip address, partition a disk 
and create an ext4 file system. 


paul@ubu1010:~$ ls -1 /sbin/ifconfig /sbin/fdisk /sbin/mkfs.ext4 
Erwt zr x I root root 97172 AMA AAA 09:56 /sbin/fdiisk 
Erwe xe x iroot root 65708 2010070210927 SEIN nte eo oues ior 
Srwxr zr x 5 root root 55140 2010-08-18 18:01 /sban/mkfis.ext4 
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11.4.4. /lib 


Binaries found in /bin and /sbin often use shared libraries located in /lib. Below is a 
screenshot of the partial contents of /lib. 


paul@laika:~$ ls /lib/libc* 


Pf Mali be 7 9, S0 /lip/libefont so: 0-0-0 (/ lab /tibcomierrso.2. 1 

/lib/libcap.so.1 ¡Año etica ESO /lib/libconsole.so.0 

¡Ay Ae o rb 5becdne sob /lib/libconsole.so.0.0.0 

"aba eRombesoW) ¡o IE. One so ¡A IE CSV 2:550 
/lib/modules 


Typically, the Linux kernel loads kernel modules from /lib/modules/$kernel-version/. 
This directory is discussed in detail in the Linux kernel chapter. 


/lib32 and /lib64 


We currently are in a transition between 32-bit and 64-bit systems. Therefore, you may 
encounter directories named /lib32 and /lib64 which clarify the register size used during 
compilation time of the libraries. A 64-bit computer may have some 32-bit binaries and 
libraries for compatibility with legacy applications. This screenshot uses the file utility to 
demonstrate the difference. 


paul@laika:~$ file /11b32/libc—-2.5.so0 

/1p32/1l1ibe2:5:S0: ELE 32 = put LSB shared object, Intel 80386, \ 
version 1 (SYSV), for GNU/Linux 2.6.0, stripped 

paul@laika:~$ file /lib64/libcap.so.1.10 

/lib64/1libcap.so.1.10: ELF 64-bit LSB shared object, AMD x86-64, \ 
version 1 (SYSV), stripped 


The ELF (Executable and Linkable Format) is used in almost every Unix-like operating 
system since System V. 


11.4.5. /opt 


The purpose of /opt is to store optional software. In many cases this is software from outside 
the distribution repository. You may find an empty /opt directory on many systems. 


A large package can install all its files in /bin, /lib, /etc subdirectories within /opt/ 


$packagename/. If for example the package is called wp, then it installs in /opt/wp, putting 
binaries in /opt/wp/bin and manpages in /opt/wp/man. 
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11.5. configuration directories 
11.5.1. /boot 


The /boot directory contains all files needed to boot the computer. These files don't change 
very often. On Linux systems you typically find the /boot/grub directory here. /boot/grub 
contains /boot/grub/grub.cfg (older systems may still have /boot/grub/grub.conf) which 
defines the boot menu that is displayed before the kernel starts. 


11.5.2. /etc 


All of the machine-specific configuration files should be located in /etc. Historically /etc 
stood for etcetera, today people often use the Editable Text Configuration backronym. 


Many times the name of a configuration files is the same as the application, daemon, or 
protocol with .conf added as the extension. 


paul@laika:~$ ls /etc/*.conf 


/etc/adduser.conf /etc/ld.so.conf /etc/scrollkeeper.conf 
/etc/brltty.conf 7etc/ ftp. cont /etc/sysctl.conf 
/etc/ccertificates.conf  /etc/libao.conf /etc/syslog.conf 
jetc/cvs—-cron.cont /etc/logrotate.conf /etc/uct cont 
/etc/ddclient.conf /etc/ltrace.conf /etc/uniconf.conf 
/etc/debconf.conf /etc/mke2fs.conf /etc/updatedb.conf 
/etc/deluser.conf /etc/netscsid.conf /etc/usplash.conf 
/etc/fdmount.conf /etc/nsswitch.conf /etc/uswsusp.conf 
/etc/hdparm.conf /etc/pam.conf /etc/vnc.conf 
/etc/host.conf /etc/pnm2ppa.conf /etc/wodim.conf 
/etc/inetd.conf /etc/povray.conf /etc/wvdial.conf 
/etc/kernel-img.conf /etc/resolv.conf 


paulflaika:-S 


There is much more to be found in /etc. 
letc/init.d/ 


A lot of Unix/Linux distributions have an /etc/init.d directory that contains scripts to start 
and stop daemons. This directory could disappear as Linux migrates to systems that replace 
the old init way of starting all daemons. 


letc/X11/ 


The graphical display (aka X Window System or just X) is driven by software from the 
X.org foundation. The configuration file for your graphical display is /etc/X11/xorg.conf. 


/etc/skel/ 


The skeleton directory /etc/skel is copied to the home directory of a newly created user. It 
usually contains hidden files like a .bashre script. 


letc/sysconfig/ 


This directory, which is not mentioned in the FHS, contains a lot of Red Hat Enterprise 
Linux configuration files. We will discuss some of them in greater detail. The screenshot 
below is the /etc/sysconfig directory from RHELv4u4 with everything installed. 
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paul@RHELv4u4:~$ ls /etc/sysconfig/ 


apmd firstboot irda network saslauthd 
apm-scripts grub irgbalance networking selinux 
authconfig hidd keyboard ntpd spamassassin 
autofs httpd kudzu openib.conf squid 
bluetooth hwconf lm sensors pand syslog 

clock i18n mouse pcmcia sys-config-sec 
console init mouse. B pgsql sys-config-users 
crond installinfo named prelink sys-logviewer 
desktop ipmi netdump rawdevices tux 

diskdump iptables netdump_id_dsa rhn vncservers 
dund iptables-cfg  netdump id dsa.p samba xinetd 


paul@RHELv4u4:~$ 


The file /etc/sysconfig/firstboot tells the Red Hat Setup Agent not to run at boot time. If 
you want to run the Red Hat Setup Agent at the next reboot, then simply remove this file, 
and run chkconfig --level 5 firstboot on. The Red Hat Setup Agent allows you to install 
the latest updates, create a user account, join the Red Hat Network and more. It will then 
create the /etc/sysconfig/firstboot file again. 


paul@RHELv4u4:~$ cat /etc/sysconfig/firstboot 
RUN FIRSTBOOTZNO 


The /etc/sysconfig/harddisks file contains some parameters to tune the hard disks. The file 
explains itself. 


You can see hardware detected by kudzu in /etc/sysconfig/hwconf. Kudzu is software from 
Red Hat for automatic discovery and configuration of hardware. 


The keyboard type and keymap table are set in the /etc/sysconfig/keyboard file. For more 
console keyboard information, check the manual pages of keymaps(5), dumpkeys(1), 
loadkeys(1) and the directory /lib/kbd/keymaps/. 


root @RHELv4u4: /etc/sysconfig# cat keyboard 
KEYBOARDTYPE="pc" 
KEYTABLE="us" 


We will discuss networking files in this directory in the networking chapter. 
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11.6. data directories 
11.6.1. /home 


Users can store personal or project data under /home. It is common (but not mandatory by 
the fhs) practice to name the users home directory after the user name in the format /home/ 
$USERNAME. For example: 


paul@ubu606:~$ ls /home 
geert annik sandra paul tom 


Besides giving every user (or every project or group) a location to store personal files, the 
home directory of a user also serves as a location to store the user profile. A typical Unix 
user profile contains many hidden files (files whose file name starts with a dot). The hidden 
files of the Unix user profiles contain settings specific for that user. 


paul@ubu606:~$ ls -d /home/paul/.* 

/home/paul/. /home/paul/.bash_profile /home/paul/.ssh 
/home/paul/.. /home/paul/.bashrc /home/paul/.viminfo 
/home/paul/.bash history  /home/paul/.lesshst 


11.6.2. /root 


On many systems /root is the default location for personal data and profile of the root user. 
If it does not exist by default, then some administrators create it. 


11.6.3. /srv 


You may use /srv for data that is served by your system. The FHS allows locating cvs, 
rsync, ftp and www data in this location. The FHS also approves administrative naming in / 
srv, like /srv/project55/ftp and /srv/sales/www. 


On Sun Solaris (or Oracle Solaris) /export is used for this purpose. 
11.6.4. /media 


The /media directory serves as a mount point for removable media devices such as CD- 
ROM's, digital cameras, and various usb-attached devices. Since /media is rather new in the 
Unix world, you could very well encounter systems running without this directory. Solaris 
9 does not have it, Solaris 10 does. Most Linux distributions today mount all removable 
media in /media. 


paul@debian5:~$ ls /media/ 
cdrom cdrom0O usbdisk 


11.6.5. /mnt 


The /mnt directory should be empty and should only be used for temporary mount points 
(according to the FHS). 
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Unix and Linux administrators used to create many directories here, like /mnt/something/. 
You likely will encounter many systems with more than one directory created and/or 
mounted inside /mnt to be used for various local and remote filesystems. 


11.6.6. /tmp 


Applications and users should use /tmp to store temporary data when needed. Data stored 
in /tmp may use either disk space or RAM. Both of which are managed by the operating 
system. Never use /tmp to store data that is important or which you wish to archive. 
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11.7. in memory directories 
11.7.1. /dev 


Device files in /dev appear to be ordinary files, but are not actually located on the hard disk. 
The /dev directory is populated with files as the kernel is recognising hardware. 


common physical devices 


Common hardware such as hard disk devices are represented by device files in /dev. Below 
a screenshot of SATA device files on a laptop and then IDE attached drives on a desktop. 
(The detailed meaning of these devices will be discussed later.) 


+ 

# SATA or SCSI or USB 

++ 

paul@laika:~$ ls /dev/sd* 

/dev/sda /dev/sdal /dev/sda2 /dev/sda3 /dev/sdb /dev/sdbl /dev/sdb2 


# 

# IDE or ATAPI 

# 

paul@barry:~$ ls /dev/hd* 

/dev/hda /dev/hdal /dev/hda2 /dev/hdb /dev/hdbl /dev/hdb2 /dev/hdc 


Besides representing physical hardware, some device files are special. These special devices 
can be very useful. 


/dev/tty and /dev/pts 


For example, /dev/ttyl represents a terminal or console attached to the system. (Don't 
break your head on the exact terminology of 'terminal' or 'console', what we mean here is 
a command line interface.) When typing commands in a terminal that is part of a graphical 
interface like Gnome or KDE, then your terminal will be represented as /dev/pts/1 (1 can 
be another number). 


/dev/null 


On Linux you will find other special devices such as /dev/null which can be considered 
a black hole; it has unlimited storage, but nothing can be retrieved from it. Technically 
speaking, anything written to /dev/null will be discarded. /dev/null can be useful to discard 
unwanted output from commands. /dev/null is not a good location to store your backups ;-). 


11.7.2. /proc conversation with the kernel 


/proc is another special directory, appearing to be ordinary files, but not taking up disk 
space. It is actually a view of the kernel, or better, what the kernel manages, and is a means 
to interact with it directly. /proc is a proc filesystem. 


paul@RHELv4u4:~$ mount -t proc 
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none on /proc type proc 


(rw) 


When listing the /proc directory you will see many numbers (on any Unix) and some 


interesting files (on Linux) 


mul@laika:~$ ls /proc 


al 2339 4724 5418 
10175 2523 4729 5421 
OZ ASS 4741 5658 
1102392975 4873 5661 
141 29775. 4874 5665 
15045 29792 4878 5927 
1519 2997 4879 6 

1548 3 4881 6032 
1551 30228 4882 6033 
1554 3069 5 6145 
1557 91422 5073 26298 
1606 3149 5147 6414 
180 31507 5203 6418 
181 3189 5206 6419 
182 3193 5228 6420 
18898 3246 5272 6421 
19799: 3248 5291 6422 
19903 SP DS 5294 6423 
19804 3372 5356 6424 
1987 4 5370 6425 
1989 42 5379 6426 
2 45 5380 6430 
20845 4542 5412 6450 
221 46 5414 6551 
2338 4704 5416 6568 


7201 
7204 
7206 
7214 
7216 
WZ ls} 
7223 
7224 
722 
7260 
7267 
1295) 
7282 
7298 
TSS) 
7330 
7345 
T513 
7525 
7529 
9964 
acpi 
asound 
buddyinfo 
bus 


cmdline 
cpuinfo 
crypto 
devices 
diskstats 
dma 

driver 
execdomains 
fb 
filesystems 
fs 

ide 
interrupts 
iomem 
ioports 
irg 
kallsyms 
kcore 
key-users 
kmsg 
loadavg 
locks 
meminfo 
misc 
modules 


mounts 

MELE 

net 
pagetypeinfo 
partitions 
sched debug 
scsi 

self 
slabinfo 
stat 

swaps 

sys 
sysrq-trigger 
Sysvipc 
timer list 
timer stats 
ECY 

uptime 
version 
version_signature 
vmcore 

vmnet 

vmstat 
zoneinfo 


Let's investigate the file properties inside /proc. Looking at the date and time will display 
the current date and time showing the files are constantly updated (a view on the kernel). 


...time passes... 


~$ ls -al /proc/cpuinfo 


paul@RHELv4u4:~$ date 

Mon Jan 29 18:06:32 EST 2007 
paul@RHELv4u4:~$ ls -al /proc/cpuinfo 
Seige = il 
paul@RHELv4u4:~$ 
paul@RHELv4u4:~$ 
paul@RHELv4u4:~$ 
paul@RHELv4u4:~$ date 

Mon Jan 29 18:10:00 EST 2007 
paul@RHELv4ué4: 

== eek 


root root 0 Jan 29 18:06 /proc/cpuinfo 


root root 0 Jan 29 18:10 o6 oc cpuin fo 


Most files in /proc are 0 bytes, yet they contain data--sometimes a lot of data. You can see 
this by executing cat on files like /proc/cpuinfo, which contains information about the CPU. 


paul@RHELv4u4:~$ file /proc/cpuinfo 


/proc/cpuinfo: empty 
paul@RHELv4u4:~$ cat /proc/cpuinfo 
processor 2-10) 

vendor id AuthenticAMD 

cpu family SS 

model : 43 
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model name : AMD Athlon(tm) 64 X2 Dual Core Processor 4600+ 
stepping Sox 

cpu MHz : 2998.628 

cache size $: 512 KB 

fdiv bug O 

hlt bug O 

f00f bug LO 

coma_bug O 

fpu : yes 

fpu exception : yes 

cpuid level sub 

wp : yes 

flags : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge... 
bogomips + 4803.54 


Just for fun, here is /proc/cpuinfo on a Sun Sunblade 1000... 


paul@pasha:~$ cat /proc/cpuinfo 
cpu : TI UltraSparc III (Cheetah) 
fpu : UltraSparc III integrated FPU 
promlib : Version 3 Revision 2 
prom k 4.22 

type : sun4u 

ncpus probed : 2 

ncpus active : 2 

Cpu0Bogo : 498.68 

CpuOClkTck : 000000002cb41780 
CpulBogo : 498.68 

CpulClkTck : 000000002cb41780 
MMU Type : Cheetah 

States 

CPUO: online 

CPU1: online 


Most of the files in /proc are read only, some require root privileges, some files are writable, 
and many files in /proc/sys are writable. Let's discuss some of the files in /proc. 
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/proc/interrupts 


On the x86 architecture, /proc/interrupts displays the interrupts. 


paul@RHELv4u4:~$ cat /proc/interrupts 


CPUO 
0 US TOS 7/ 3) IO-APIC-edg timer 
I 15 IO-APIC-edge i8042 
Se al IO-APIC-edge rtc 
Sis 0 IO-APIC-level acpi 
Jig 8 67 IO-APIC-edge i8042 
14: 128 IO-APIC-edge  ideO 
Sys 124320 IO-APIC-edge idel 
TOSE 11993 IO-APIC-level ioc0 
I 2428 IO-APIC-level eth0 
NMI: 0 
LOC: 13878037 
ERR: 0 
MIS: 0 


On a machine with two CPU's, the file looks like this. 


paul@laika:~$ cat /proc/interrupts 


CPUO CPU1 
0 860013 O IO-APIC-edge timer 
T3 4533 0 IO-APIC-edge i8042 
Ta 0 O IOC-APIC-edge parportO0 
8: 6588227 O IO-APIC-edge TEG 
qos 2314 0 IO-APIC-fasteoi acpi 
ig 135 O IO-APIC-edge 18042 
14: 0 O IOC-APIC-edge libata 
153 12209 O IO-APIC-edge libata 
es dl O IO-APIC-fasteoi yenta 
LO 115036 O IO-APIC-fasteoi  ethO 
206 126871 0 IO-APIC-fasteoi libata, ohcil394 
uds 30204 O IO-APIC-fasteoi ehci_hcd:usbl, uhci hcd:usb2 
22 1334 Q IO-APIC-fasteoi  saa7133[0], saa7133[0] 
24: 234739 Q IO-APIC-fasteoi nvidia 
NMI: 72 42 
noces 860000 859994 
ERR: 0 
/proc/kcore 


The physical memory is represented in /proc/kcore. Do not try to cat this file, instead use a 
debugger. The size of /proc/kcore is the same as your physical memory, plus four bytes. 
paul@laika:~$ ls -Ih /proc/kcore 


SPSS aaa I root root 2:06 2007-01-30 08:57 /proc/kcore 
paul@laika:~$ 
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11.7.3. /sys Linux 2.6 hot plugging 


The /sys directory was created for the Linux 2.6 kernel. Since 2.6, Linux uses sysfs 
to support usb and IEEE 1394 (FireWire) hot plug devices. See the manual pages 
of udev(8) (the successor of devfs) and hotplug(8) for more info (or visit http://linux- 
hotplug.sourceforge.net/ ). 


Basically the /sys directory contains kernel information about hardware. 


113 


the Linux file tree 


11.8. /usr Unix System Resources 


Although /usr is pronounced like user, remember that it stands for Unix System Resources. 
The /usr hierarchy should contain shareable, read only data. Some people choose to mount 
/usr as read only. This can be done from its own partition or from a read only NFS share 
(NFS is discussed later). 


11.8.1. /usr/bin 


The /usr/bin directory contains a lot of commands. 


paul@deb508:~$ ls /usr/bin | wc -1 
1395 


(On Solaris the /bin directory is a symbolic link to /usr/bin.) 
11.8.2. /usr/include 


The /usr/include directory contains general use include files for C. 


paul@ubu1010:~$ 1s /usr/include/ 


aalib.h expat config.h math.h search.h 
atovis oh expat_external.h mcheck.h semaphore.h 
aio.h expat.h memory.h setjmp.h 
AL fentl -h menu.h sgtty.h 
aliases.h features.h mntent.h shadow.h 


11.8.3. /usr/lib 


The /usr/lib directory contains libraries that are not directly executed by users or scripts. 


paul@deb508:~$ ls /usr/lib | head -7 
4Suite 

ao 

apt 

arj 

aspell 

avahi 

bonobo 


11.8.4. /usr/local 


The /usr/local directory can be used by an administrator to install software locally. 
paul@deb508:~$ 1s /usr/local/ 
bin etc games include lib man sbin share src 


paul@deb508:~$ du -sh /usr/local/ 
128K /usr/local/ 


11.8.5. /usr/share 


The /usr/share directory contains architecture independent data. As you can see, this is a 
fairly large directory. 


paul@deb508:~$ ls /usr/share/ | wc -1 
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263 
paul@deb508:~$ du -sh /usr/share/ 
1.3G /usr/share/ 


This directory typically contains /usr/share/man for manual pages. 


paul@deb508:~$ ls /usr/share/man 


ES fr hu it.UTF-8 man2 man6 pl.1ISO8859-2 sv 
dieras OS Ia mans man?  pl.UTF-8 ¡E 
es  fr.UTF-8 ie ko man4 man8 pt_BR zh_CN 
Hes — (en it.IS08859-1 manl man5 pl ru zh TW 


And it contains /usr/share/games for all static game data (so no high-scores or play logs). 


paul@ubu1010:~$ ls /usr/share/games/ 
openttd  wesnoth 


11.8.6. /usr/src 


The /usr/sre directory is the recommended location for kernel source files. 


paul@deb508:~$ ls -1 /usr/src/ 

total 12 

drwxr-xr-x 4 root root 4096 2011-02-01 14:43 linux-headers-2.6.26-2-686 
drwxr-xr-x 18 root root 4096 2011-02-01 14:43 linux-headers-2.06.26-2—common 
drwxr-xr-x 3 root root 4096 2009-10-28 16:01 linux-kbuild-2.6.26 
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11.9. /var variable data 


Files that are unpredictable in size, such as log, cache and spool files, should be located in 
/var. 


11.9.1. /var/log 


The /var/log directory serves as a central point to contain all log files. 


[paul@RHEL4b ~]$ ls /var/log 


acpid Gron. 2 maillog.2 quagga secure.4 
amanda Cron. 3 maillog.3 radius spooler 
anaconda.log cron.4 maillog.4 rpmpkgs spooler.1 
anaconda.syslog cups mailman rpmpkgs.1 spooler.2 
anaconda.xlog dmesg messages rpmpkgs.2 Spooler.3 
audit exim messages.l  rpmpkgs.3 spooler.4 
boot.log gdm messages.2  rpmpkgs.4 squid 

book. Log. T httpd messages.3 sa uucp 

POE log. 2 iiim messages.4 samba vbox 

boot. Llog.3 iptraf mysqld.log scrollkeeper.log vmware-tools-guestd 
boot.log.4 lastlog news secure wtmp 

canna mail pgsql secure.1 wtmp.1 

cron maillog ppp secure.2 Xorg. tilo gs; 
cron. maillog.1 prelink.log secure.3 Xorg. 0. log. old 


11.9.2. /var/log/messages 


A typical first file to check when troubleshooting on Red Hat (and derivatives) is the /var/ 
log/messages file. By default this file will contain information on what just happened to the 
system. The file is called /var/log/syslog on Debian and Ubuntu. 


[root@RHEL4b ~]# tail /var/log/messages 

1 30 05:13:56 anacron: anacron startup succeeded 

L 30 05:13:56 atd: atd startup succeeded 

1 30 05:13:57 messagebus: messagebus startup succeeded 

| 30 05:13:57 cups-config-daemon: cups-config-daemon startup succeeded 
| 30 05:13:58 haldaemon: haldaemon startup succeeded 

L 30 05:14:00 fstab-sync[3560]: removed all generated mount points 

L 30 05:14:01 fstab-sync[3628]: added mount point /media/cdrom for... 
L 30 05:14:01 fstab-sync[3646]: added mount point /media/floppy for... 
L 30 05:16:46 sshd(pam unix) [3662]: session opened for user paul by... 
| 30 06:06:37 su(pam unix) [3904]: session opened for user root by paul 


E 
ae 


11.9.3. /var/cache 


The /var/cache directory can contain cache data for several applications. 


paul@ubu1010:~$ ls /var/cache/ 


apt dictionaries-common gdm man software-center 
binfmts  flashplugin-installer hald pm-utils 

cups fontconfig jockey pppconfig 

debconf fonts ldconfig samba 


11.9.4. /var/spool 


The /var/spool directory typically contains spool directories for mail and cron, but also 
serves as a parent directory for other spool files (for example print spool files). 


116 


the Linux file tree 


11.9.5. /var/lib 


The /var/lib directory contains application state information. 


Red Hat Enterprise Linux for example keeps files pertaining to rpm in /var/lib/rpm/. 
11.9.6. /var/... 


/var also contains Process ID files in /var/run (soon to be replaced with /run) and temporary 
files that survive a reboot in /var/tmp and information about file locks in /var/lock. There 
will be more examples of /var usage further in this book. 
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11.10. practice: file system tree 


1. Does the file /bin/cat exist ? What about /bin/dd and /bin/echo. What is the type of these 
files ? 


2. What is the size of the Linux kernel file(s) (vmlinu*) in /boot ? 


3. Create a directory ~/test. Then issue the following commands: 
Giotto 


dd if-/dev/zero of-zeroes.txt count-1 bs-100 


od zeroes.txt 


dd will copy one times (count=1) a block of size 100 bytes (bs=100) from the file /dev/zero 
to ~/test/zeroes.txt. Can you describe the functionality of /dev/zero ? 


4. Now issue the following command: 


dd if-/dev/random of-random.txt count-1 bs-100 ; od random.txt 


dd will copy one times (count=1) a block of size 100 bytes (bs=100) from the file /dev/ 
random to ~/test/random.txt. Can you describe the functionality of /dev/random ? 


5. Issue the following two commands, and look at the first character of each output line. 
ls -1 /dev/sd* /dev/hd* 


ls -1 /dev/tty* /dev/input/mou* 


The first Is will show block(b) devices, the second Is shows character(c) devices. Can you 
tell the difference between block and character devices ? 


6. Use cat to display /etc/hosts and /etc/resolv.conf. What is your idea about the purpose 
of these files ? 


7. Are there any files in /etc/skel/ ? Check also for hidden files. 

8. Display /proc/cpuinfo. On what architecture is your Linux running ? 

9. Display /proc/interrupts. What is the size of this file ? Where is this file stored ? 
10. Can you enter the /root directory ? Are there (hidden) files ? 


11. Are ifconfig, fdisk, parted, shutdown and grub-install present in /sbin ? Why are these 
binaries in /sbin and not in /bin ? 


12. Is /var/log a file or a directory ? What about /var/spool ? 
13. Open two command prompts (Ctrl-Shift- T in gnome-terminal) or terminals (Ctrl-Alt-F1, 


Ctrl-Alt-F2, ...) and issue the who am i in both. Then try to echo a word from one terminal 
to the other. 
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14. Read the man page of random and explain the difference between /dev/random and / 
dev/urandom. 
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11.11. solution: file system tree 
1. Does the file /bin/cat exist ? What about /bin/dd and /bin/echo. What is the type of these 
files ? 
Is /pin/cat ~ file pia cat 
ls /bin/dd ; file /bin/dd 
ls /bin/echo ; file /bin/echo 
2. What is the size of the Linux kernel file(s) (vmlinu*) in /boot ? 


ls -lh /boot/vm* 


3. Create a directory ~/test. Then issue the following commands: 


cd ~/test 


dd if-/dev/zero of-zeroes.txt count-1 bs-100 


od zeroes.txt 


dd will copy one times (count=1) a block of size 100 bytes (bs=100) from the file /dev/zero 
to ~/test/zeroes.txt. Can you describe the functionality of /dev/zero ? 


/dev/zero is a Linux special device. It can be considered a source of zeroes. You cannot send 
something to /dev/zero, but you can read zeroes from it. 


4. Now issue the following command: 


dd if-/dev/random of-random.txt count-1 bs-100 ; od random.txt 


dd will copy one times (count=1) a block of size 100 bytes (bs=100) from the file /dev/ 
random to ~/test/random.txt. Can you describe the functionality of /dev/random ? 


/dev/random acts as a random number generator on your Linux machine. 


5. Issue the following two commands, and look at the first character of each output line. 
ls -1 /dev/sd* /dev/hd* 


ls -1 /dev/tty* /dev/input/mou* 


The first Is will show block(b) devices, the second Is shows character(c) devices. Can you 
tell the difference between block and character devices ? 


Block devices are always written to (or read from) in blocks. For hard disks, blocks of 512 
bytes are common. Character devices act as a stream of characters (or bytes). Mouse and 
keyboard are typical character devices. 


6. Use cat to display /etc/hosts and /etc/resolv.conf. What is your idea about the purpose 
of these files ? 


/etc/hosts contains hostnames with their ip address 


/etc/resolv.conf should contain the ip address of a DNS name server. 
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7. Are there any files in /etc/skel/ ? Check also for hidden files. 


Issue "ls -al /etc/skel/". Yes, there should be hidden files there. 


8. Display /proc/cpuinfo. On what architecture is your Linux running ? 


The file should contain at least one line with Intel or other cpu. 
9. Display /proc/interrupts. What is the size of this file ? Where is this file stored ? 


The size is zero, yet the file contains data. It is not stored anywhere because /proc is a 
virtual file system that allows you to talk with the kernel. (If you answered "stored in RAM- 
memory, that is also correct...). 


10. Can you enter the /root directory ? Are there (hidden) files ? 


Try "cd /root". The /root directory is not accessible for normal users on most modern Linux sy 


11. Are ifconfig, fdisk, parted, shutdown and grub-install present in /sbin ? Why are these 
binaries in /sbin and not in /bin ? 


Because those files are only meant for system administrators. 


12. Is /var/log a file or a directory ? What about /var/spool ? 


Both are directories. 


13. Open two command prompts (Ctrl-Shift- T in gnome-terminal) or terminals (Ctrl-Alt-F1, 
Ctrl-Alt-F2, ...) and issue the who am i in both. Then try to echo a word from one terminal 
to the other. 


tty-terminal: echo Hello > /dev/ttyl 


pts-terminal: echo Hello » /dev/pts/1 


14. Read the man page of random and explain the difference between /dev/random and / 
dev/urandom. 


man 4 random 
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Chapter 12. commands and 
arguments 


This chapter introduces you to shell expansion by taking a close look at commands and 
arguments. Knowing shell expansion is important because many commands on your 
Linux system are processed and most likely changed by the shell before they are executed. 


The command line interface or shell used on most Linux systems is called bash, which 
stands for Bourne again shell. The bash shell incorporates features from sh (the original 
Bourne shell), csh (the C shell), and ksh (the Korn shell). 


This chapter frequently uses the echo command to demonstrate shell features. The echo 
command is very simple: it echoes the input that it receives. 

paul@laika:~$ echo Burtonville 

Burtonville 


paul@laika:~$ echo Smurfs are blue 
Smurfs are blue 
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12.1. arguments 


One of the primary features of a shell is to perform a command line scan. When you enter 
a command at the shell's command prompt and press the enter key, then the shell will start 
scanning that line, cutting it up in arguments. While scanning the line, the shell may make 
many changes to the arguments you typed. 


This process is called shell expansion. When the shell has finished scanning and modifying 
that line, then it will be executed. 


12.2. white space removal 


Parts that are separated by one or more consecutive white spaces (or tabs) are considered 
separate arguments, any white space is removed. The first argument is the command to be 
executed, the other arguments are given to the command. The shell effectively cuts your 
command into one or more arguments. 


This explains why the following four different command lines are the same after shell 
expansion. 


[paul@RHELv4u3 ~]$ echo Hello World 

Hello World 

[paul@RHELv4u3 ~]$ echo Hello World 

Hello World 

[paul@RHELv4u3 -]$ echo Hello World 

Hello World 

[paul@RHELv4u3 ~]$ echo Hello World 
Hello World 


The echo command will display each argument it receives from the shell. The echo 
command will also add a new white space between the arguments it received. 
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12.3. single quotes 


You can prevent the removal of white spaces by quoting the spaces. The contents of the 
quoted string are considered as one argument. In the screenshot below the echo receives 
only one argument. 


[paul@RHEL4b ~]$ echo 'A line with single quotes' 
A line with single quotes 
[paul@RHEL4b -]$ 


12.4. double quotes 


You can also prevent the removal of white spaces by double quoting the spaces. Same as 
above, echo only receives one argument. 


[paul@RHEL4b -]$ echo "A line with double quotes" 
A line with double quotes 
[paul@RHEL4b -]$ 


Later in this book, when discussing variables we will see important differences between 
single and double quotes. 


12.5. echo and quotes 


Quoted lines can include special escaped characters recognised by the echo command (when 
using echo -e). The screenshot below shows how to use WM for a newline and X for a tab 
(usually eight white spaces). 


paul@RHEL4b ~]$ echo -e "A line with Mna newline" 
A line with 

a newline 

paul@RHEL4b ~]$ echo -e 'A line with Ina newline' 
A line with 

a newline 

paul@RHEL4b ~]$ echo -e "A line with \ta tab" 
A line with a tab 

paul@RHEL4b ~]$ echo -e 'A line with \ta tab' 
A line with a tab 

paul@RHEL4b ~]$ 


The echo command can generate more than white spaces, tabs and newlines. Look in the 
man page for a list of options. 
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12.6. commands 


12.6.1. external or builtin commands ? 


Not all commands are external to the shell, some are builtin. External commands are 
programs that have their own binary and reside somewhere in the file system. Many external 
commands are located in /bin or /sbin. Builtin commands are an integral part of the shell 
program itself. 


12.6.2. type 


To find out whether a command given to the shell will be executed as an external command 
or as a builtin command, use the type command. 


paul@laika:~$ type cd 
cd is a shell builtin 
paul@laika:~$ type cat 
cat is /bin/cat 


As you can see, the cd command is builtin and the cat command is external. 


You can also use this command to show you whether the command is aliased or not. 


paul@laika:~S$ type ls 
ls is aliased to “ls --color-auto' 


12.6.3. running external commands 


Some commands have both builtin and external versions. When one of these commands is 
executed, the builtin version takes priority. To run the external version, you must enter the 
full path to the command. 


paul@laika:~$ type -a echo 

echo is a shell builtin 

echo is /bin/echo 

paul@laika:~$ /bin/echo Running the external echo command... 
Running the external echo command... 


12.6.4. which 


The which command will search for binaries in the $PATH environment variable (variables 
will be explained later). In the screenshot below, it is determined that cd is builtin, and ls, 
cp, rm, mv, mkdir, pwd, and which are external commands. 


[root@RHEL4b ~]# which cp ls cd mkdir pwd 

VIDE VAGOS 

/bin/ls 

/usr/bin/which: no cd in (/usr/kerberos/sbin:/usr/kerberos/bin:... 
/bin/mkdir 

/bin/pwd 
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12.7. aliases 


12.7.1. create an alias 


The shell allows you to create aliases. Aliases are often used to create an easier to remember 
name for an existing command or to easily supply parameters. 


[paul@RHELv4u3 ~]$ cat count.txt 
one 

two 

three 

[paul@RHELv4u3 ~]$ alias dog=tac 
[paul@RHELv4u3 ~]$ dog count.txt 
three 

two 

one 


12.7.2. abbreviate commands 


An alias can also be useful to abbreviate an existing command. 


paul@laika:~$ alias 11='1s -1h color=auto' 
paul@laika:~$ alias c='clear' 
paul@laika:~s 


12.7.3. default options 


Aliases can be used to supply commands with default options. The example below shows 
how to set the -i option default when typing rm. 


paul@RHELv4u3 ~]$ rm -i winter.txt 

rm: remove regular file ^winter.txt'? no 
paul@RHELv4u3 ~]$ rm winter.txt 

paul@RHELv4u3 -]$ ls winter.txt 

Us: winter.txt: No such tile or directory 
paul@RHELv4u3 ~]$ touch winter.txt 
paul@RHELv4u3 ~]$ alias rm='rm -i' 
paul@RHELv4u3 ~]$ rm winter.txt 

rm: remove regular empty file 'winter.txt'? no 
paul@RHELv4u3 -]$ 


Some distributions enable default aliases to protect users from accidentally erasing files (‘rm 
-1', 'mv -1', 'cp -1') 


12.7.4. viewing aliases 
You can provide one or more aliases as arguments to the alias command to get their 
definitions. Providing no arguments gives a complete list of current aliases. 
paul@laika:~$ alias c 11 


alias c='clear' 
alias ll-'l1s -1h color-auto' 


129 


commands and arguments 


12.7.5. unalias 


You can undo an alias with the unalias command. 


paul@RHEL4b ~]$ which rm 
/bin/rm 
paul@RHEL4b ~]$ alias rm='rm -i' 
paul@RHEL4b ~]$ which rm 
alias rm='rm -i' 

/bin/rm 
paul@RHEL4b -]$ unalias rm 
paul@RHEL4b ~]$ which rm 
/bin/rm 
paul@RHEL4b ~]$ 


12.8. displaying shell expansion 


You can display shell expansion with set -x, and stop displaying it with set +x. You might 
want to use this further on in this course, or when in doubt about exactly what the shell is 
doing with your command. 


[paul@RHELv4u3 -]$ set -x 

++ echo ne '\033]0;paul@RHELv4u3:~\007' 
[paul@RHELv4u3 -]$ echo SUSER 
+ echo paul 
paul 

++ echo ne '\033]0;paul@RHELv4u3:~\007' 
[paul@RHELv4u3 -]$ echo \SUSER 

+ echo 'SUSER' 

SUSER 

++ echo -ne '\033]0;paul@RHELV4u3:~\007' 
[paul@RHELv4u3 -]$ set +x 

+ Set +x 

[paul@RHELv4u3 -]$ echo SUSER 

paul 


130 


commands and arguments 


12.9. practice: commands and arguments 


1. How many arguments are in this line (not counting the command itself). 


touch "7Zetc/cron/cnonwallow' “tile 42 text" "file 33. tst 
2. Is tac a shell builtin command ? 
3. Is there an existing alias for rm ? 


4. Read the man page of rm, make sure you understand the -i option of rm. Create and 
remove a file to test the -i option. 


5. Execute: alias rmz'rm -i' . Test your alias with a test file. Does this work as expected ? 
6. List all current aliases. 

7a. Create an alias called 'city' that echoes your hometown. 

7b. Use your alias to test that it works. 

8. Execute set -x to display shell expansion for every command. 

9. Test the functionality of set -x by executing your city and rm aliases. 

10 Execute set +x to stop displaying shell expansion. 

11. Remove your city alias. 

12. What is the location of the cat and the passwd commands ? 


13. Explain the difference between the following commands: 
echo 


/bin/echo 


14. Explain the difference between the following commands: 
echo Hello 


echo) n Helio 
15. Display A B C with two spaces between B and C. 


(optional)16. Complete the following command (do not use spaces) to display exactly the 
following output: 


17. Use echo to display the following exactly: 


BONN 
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Find two solutions with single quotes, two with double quotes and one without quotes (and 
say thank you to René and Darioush from Google for this extra). 


18. Use one echo command to display three words on three lines. 


132 


commands and arguments 


12.10. solution: commands and arguments 


1. How many arguments are in this line (not counting the command itself). 
touch '/etc/cron/cron.allow' 'file 42.txt' "file 33.txt" 

answer: three 

2. Is tac a shell builtin command ? 

type tac 

3. Is there an existing alias for rm ? 


alias rm 


4. Read the man page of rm, make sure you understand the -i option of rm. Create and 
remove a file to test the -i option. 


man rm 
touch testfile 


rm -i testfile 


5. Execute: alias rmz'rm -i' . Test your alias with a test file. Does this work as expected ? 
touch) testiile 


rm testfile (should ask for confirmation) 


6. List all current aliases. 


alias 


7a. Create an alias called 'city' that echoes your hometown. 


alias city='echo Antwerp' 


7b. Use your alias to test that it works. 


city (it should display Antwerp) 


8. Execute set -x to display shell expansion for every command. 


set -x 


9. Test the functionality of set -x by executing your city and rm aliases. 

shell should display the resolved aliases and then execute the command: 
paul@deb503:~S set -x 

paul@deb503:~S city 


+ echo antwerp 
antwerp 


10 Execute set +x to stop displaying shell expansion. 


set +x 


11. Remove your city alias. 
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unalias city 


12. What is the location of the cat and the passwd commands ? 
which cat (probably /bin/cat) 


which passwd (probably /usr/bin/passwd) 


13. Explain the difference between the following commands: 
echo 


/bin/echo 


The echo command will be interpreted by the shell as the built-in echo command. The /bin/ 
echo command will make the shell execute the echo binary located in the /bin directory. 


14. Explain the difference between the following commands: 
echo Hello 


echo -n Hello 


The -n option of the echo command will prevent echo from echoing a trailing newline. echo 
Hello will echo six characters in total, echo -n hello only echoes five characters. 


(The -n option might not work in the Korn shell.) 


15. Display A B C with two spaces between B and C. 


echo "AB Cc" 


16. Complete the following command (do not use spaces) to display exactly the following 
output: 


The solution is to use tabs with \t. 


echo -e "4+4\t=8" ; echo -e "104+14\t=24" 


17. Use echo to display the following exactly: 


QE 

echo '??\\' 

echo -e '??\\\\! 
echo "??\\\\" 

echo -e "??\\\\\\" 
echo ??\\\\ 


Find two solutions with single quotes, two with double quotes and one without quotes (and 
say thank you to René and Darioush from Google for this extra). 


18. Use one echo command to display three words on three lines. 


echo -e "one \ntwo \nthree" 
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In this chapter we put more than one command on the command line using control 
operators. We also briefly discuss related parameters ($?) and similar special characters(&). 
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13.1. ; semicolon 


You can put two or more commands on the same line separated by a semicolon ; . The shell 
will scan the line until it reaches the semicolon. All the arguments before this semicolon 
will be considered a separate command from all the arguments after the semicolon. Both 
series will be executed sequentially with the shell waiting for each command to finish before 
starting the next one. 


paul@RHELv4u3 -]$ echo Hello 

Hello 

paul@RHELv4u3 -]$ echo World 
World 

paul@RHELv4u3 -]$ echo Hello ; echo World 
Hello 
World 

paul@RHELv4u3 -]$ 


13.2. & ampersand 


When a line ends with an ampersand &, the shell will not wait for the command to finish. 
You will get your shell prompt back, and the command is executed in background. You will 
get a message when this command has finished executing in background. 


[paul@RHELv4u3 ~]$ sleep 20 & 

ab) 7925 

[paul@RHELv4u3 ~]$ 

..wait 20 seconds... 

[paul@RHELv4u3 ~]$ 

[1]* Done sleep 20 


The technical explanation of what happens in this case is explained in the chapter about 
processes. 


13.3. $? dollar question mark 


The exit code of the previous command is stored in the shell variable $?. Actually $? is a 
shell parameter and not a variable, since you cannot assign a value to $?. 


paul@debian5:~/test$ touch filel 
paul@debian5:~/test$ echo $? 


paul@debian5:~/test$ rm filel 
paul@debian5:~/test$ echo $? 


paul@debian5:~/test$ rm filel 
rm: cannot remove ^filel': No such file or directory 
paul@debian5:~/test$ echo $? 


paul@debian5:~/test$ 
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13.4. && double ampersand 


The shell will interpret && as a logical AND. When using && the second command is 
executed only if the first one succeeds (returns a zero exit status). 


paul@barry:~$ echo first && echo second 
first 

second 

paul@barry:~$ zecho first && echo second 
-bash: zecho: command not found 


Another example of the same logical AND principle. This example starts with a working cd 
followed by Is, then a non-working ed which is not followed by Is. 


[paul@RHELv4u3 -]$ cd gen && ls 

filel file3 File55 fileab FileAB fileabc 
file2 File4 FileA Fileab fileab2 
[paul@RHELv4u3 gen]$ cd gen && ls 

-bash: cd: gen: No such file or directory 


13.5. || double vertical bar 


The || represents a logical OR. The second command is executed only when the first 
command fails (returns a non-zero exit status). 


paul@barry:~$ echo first || echo second ; echo third 
feine srt 

third 

paul@barry:~$ zecho first | | echo second ; echo third 
-bash: zecho: command not found 

second 

third 

paul@barry:~$ 


Another example of the same logical OR principle. 


[paul@RHELv4u3 ~]$ cd gen | | ls 

[paul@RHELv4u3 gen]$ cd gen | | ls 

-bash: cd: gen: No such file or directory 
filel file3 File55 fileab FileAB fileabc 
file2 File4 FileA Fileab fileab2 


13.6. combining && and || 


You can use this logical AND and logical OR to write an if-then-else structure on the 
command line. This example uses echo to display whether the rm command was successful. 


paul@laika:~/test$ rm filel && echo It worked! || echo It failed! 
It worked! 

paul@laika:~/test$ rm filel && echo It worked! || echo It failed! 
rm: cannot remove ^filel': No such file or directory 

ihe failed! 


paul@laika:~/test$ 
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13.7. # pound sign 


Everything written after a pound sign (#) is ignored by the shell. This is useful to write a 
shell comment, but has no influence on the command execution or shell expansion. 


paul@debian4:~$ mkdir test # we create a directory 
paul@debian4:~$ cd test #### we enter the directory 
paul@debian4:~/test$ ls # is it empty ? 


paul@debian4:~/test$ 


13.8. \ escaping special characters 


The backslash \ character enables the use of control characters, but without the shell 
interpreting it, this is called escaping characters. 


[paul@RHELv4u3 ~]$ echo hello \; world 

hello ; world 

[paulGRHELv4u3 ~]$ echo hello\ Y \ world 

hello world 

[paulGRHELv4u3 ~]$ echo escaping \\\ \#\ N&N \"\ \! 
escaping NV + & " ' 

[paulGRHELv4u3 ~]$ echo escaping \\\?\*\"\! 
escaping \?*"! 


13.8.1. end of line backslash 


Lines ending in a backslash are continued on the next line. The shell does not interpret the 
newline character and will wait on shell expansion and execution of the command line until 
a newline without backslash is encountered. 


[paul@RHEL4b ~]$ echo This command line \ 
> is split in three \ 

> parts 

This command line is split in three parts 
[paul@RHEL4b -]$ 
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13.9. practice: control operators 


0. Each question can be answered by one command line! 
1. When you type passwd, which file is executed ? 

2. What kind of file is that ? 

3. Execute the pwd command twice. (remember 0.) 

4. Execute Is after cd /etc, but only if ed /ete did not error. 
5. Execute cd /etc after cd etc, but only if ed etc fails. 


6. Echo it worked when touch test42 works, and echo it failed when the touch failed. All 
on one command line as a normal user (not root). Test this line in your home directory and 
in /bin/ . 


7. Execute sleep 6, what is this command doing ? 
8. Execute sleep 200 in background (do not wait for it to finish). 


9. Write a command line that executes rm file55. Your command line should print 'success' 
if file55 is removed, and print 'failed' if there was a problem. 


(optional)10. Use echo to display "Hello World with strange’ characters V * [ } ~ A 
V." (including all quotes) 
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13.10. solution: control operators 


0. Each question can be answered by one command line! 


1. When you type passwd, which file is executed ? 


which passwd 


2. What kind of file is that ? 


file /usr/bin/passwd 


3. Execute the pwd command twice. (remember 0.) 


pwd ; pwd 


4. Execute Is after cd /etc, but only if cd /ete did not error. 


cd /etc && ls 


5. Execute cd /etc after ed etc, but only if ed etc fails. 


ed etc || cd /etc 


6. Echo it worked when touch test42 works, and echo it failed when the touch failed. All 
on one command line as a normal user (not root). Test this line in your home directory and 
in /bin/ . 

paul@deb503:~$ cd ; touch test42 && echo it worked || echo it failed 

it worked 

paul@deb503:~$ cd /bin; touch test42 && echo it worked || echo it failed 


touch: cannot touch ^test42': Permission denied 
ie farled 


7. Execute sleep 6, what is this command doing ? 


pausing for six seconds 


8. Execute sleep 200 in background (do not wait for it to finish). 


sleep 200 & 


9. Write a command line that executes rm file55. Your command line should print 'success' 
if file55 is removed, and print 'failed' if there was a problem. 


rm file55 && echo success || echo failed 


(optional)10. Use echo to display "Hello World with strange' characters V * [ } ~ V 
V." (including all quotes) 


echo \"Hello World with strange\' characters NN N* N[ \} NS NNNN NL A" 
or 


echo \""Hello World with strange' characters NV * [ ) ~ NN . "N" 
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In this chapter we learn to manage environment variables in the shell. These variables are 
often needed by applications. 
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14.1. $ dollar sign 


Another important character interpreted by the shell is the dollar sign $. The shell will look 
for an environment variable named like the string following the dollar sign and replace it 
with the value of the variable (or with nothing if the variable does not exist). 


These are some examples using SHOSTNAME, SUSER, SUID, $SHELL, and $HOME. 


[paul@RHELv4u3 -]$ echo This is the SSHELL shell 

This is the /bin/bash shell 

[paul@RHELv4u3 -]$ echo This is $SHELL on computer SHOSTNAME 
This is /bin/bash on computer RHELv4u3.localdomain 
[paul@RHELv4u3 -]$ echo The userid of SUSER is SUID 

The userid of paul is 500 
[paul@RHELv4u3 -]$ echo My homedir is SHOME 
My homedir is /home/paul 


14.2. case sensitive 


This example shows that shell variables are case sensitive! 


[paul@RHELv4u3 -]$ echo Hello SUSER 
Hello paul 

[paul@RHELv4u3 ~]$ echo Hello Suser 
Hello 


14.3. creating variables 


This example creates the variable $MyVar and sets its value. It then uses echo to verify 
the value. 


[paul@RHELv4u3 gen]$ MyVar=555 
[paul@RHELv4u3 gen]$ echo $MyVar 
555 

[paul@RHELv4u3 gen]$ 
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14.4. quotes 
Notice that double quotes still allow the parsing of variables, whereas single quotes prevent 
this. 


[paul@RHELv4u3 ~]$ MyVar-555 
[paul@RHELv4u3 ~]$ echo $MyVar 


5155 

[paul@RHELv4u3 ~]$ echo "SMyVar" 
555 

[paul@RHELv4u3 ~]$ echo 'SMyVar' 
$MyVar 


The bash shell will replace variables with their value in double quoted lines, but not in single 
quoted lines. 


paul@laika:~$ city-Burtonville 
paul@laika:~$ echo "We are in $city today." 
We are in Burtonville today. 

paul@laika:~$ echo 'We are in Scity today.' 
We are in $city today. 


14.5. set 


You can use the set command to display a list of environment variables. On Ubuntu and 
Debian systems, the set command will also list shell functions after the shell variables. Use 
set | more to see the variables then. 


14.6. unset 


Use the unset command to remove a variable from your shell environment. 


[paul@RHEL4b ~]$ MyVar=8472 
[paul@RHEL4b -]$ echo $MyVar 
8472 

[paul@RHEL4b -]$ unset MyVar 
[paul@RHEL4b -]$ echo $MyVar 


[paul@RHEL4b -]$ 
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14.7. $PS1 


The $PS1 variable determines your shell prompt. You can use backslash escaped special 
characters like \u for the username or \w for the working directory. The bash manual has 
a complete reference. 


In this example we change the value of $PS1 a couple of times. 


paul@deb503:~S PS1=prompt 
prompt 

promptPS1="prompt ' 

prompt 

prompt PS12'» ' 

> 

> PS1='\u@\hs ' 

paul@deb503$ 

paul@deb503$ PS1='\u@\h:\ws'! 
paul@deb503:~$ 


To avoid unrecoverable mistakes, you can set normal user prompts to green and the root 
prompt to red. Add the following to your .bashre for a green user prompt: 


# color prompt by paul 

RED ONIS SITO Sz Sm NT" 

WHITE='\[\033[01;00m\]' 

GREEN='\[\033[01;32m\]' 

BLUE='\[\033[01;34m\]' 

export PS1-"$(debian chroot:-*($debian chroot) }SGREEN\uSWHITE@SBLUE\hSWHITE\w\$ " 
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14.8. $PATH 


The $PATH variable is determines where the shell is looking for commands to execute 
(unless the command is builtin or aliased). This variable contains a list of directories, 
separated by colons. 


[[paul@RHEL4b ~]$ echo $PATH 
/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin: 


The shell will not look in the current directory for commands to execute! (Looking for 
executables in the current directory provided an easy way to hack PC-DOS computers). If 
you want the shell to look in the current directory, then add a . at the end of your $PATH. 


[paul@RHEL4b ~]$ PATH=SPATH:. 

[paul@RHEL4b ~]$ echo $PATH 
/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:. 
[paul@RHEL4b -]$ 


Your path might be different when using su instead of su - because the latter will take on 
the environment of the target user. The root user typically has /sbin directories added to the 
$PATH variable. 


paul@RHEL3 -]$ su 
Password: 
root@RHEL3 paul]# echo $PATH 
/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin 
root@RHEL3 paul]# exit 

paul@RHEL3 ~]$ su - 
Password: 
root@RHEL3 ~]# echo $PATH 
/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin: 
root@RHEL3 ~]# 
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14.9. env 


The env command without options will display a list of exported variables. The difference 
with set with options is that set lists all variables, including those not exported to child shells. 


But env can also be used to start a clean shell (a shell without any inherited environment). 
The env -i command clears the environment for the subshell. 


Notice in this screenshot that bash will set the $SHELL variable on startup. 


[paul@RHEL4b ~]$ bash -c 'echo $SHELL $HOME SUSER' 
/bin/bash /home/paul paul 
[paul@RHEL4b ~]$ env -i bash -c 'echo $SHELL SHOME SUSER' 
/bin/bash 

[paul@RHEL4b ~]$ 


You can use the env command to set the $LANG, or any other, variable for just one instance 
of bash with one command. The example below uses this to show the influence of the 
$LANG variable on file globbing (see the chapter on file globbing). 


[paul@RHEL4b test]$ env LANG=C bash -c 'ls File[a-z]' 
Filea Fileb 
[paul@RHEL4b test]$ env LANG-en US.UTF-8 bash -c 'ls File[a-z]' 
Filea FileA Fileb FileB 

[paul@RHEL4b test]$ 


14.10. export 


You can export shell variables to other shells with the export command. This will export 
the variable to child shells. 


[paul@RHEL4b -]$ var3-three 
[paul@RHEL4b ~]$ var4-four 
[paul@RHEL4b ~]$ export var4 
[paul@RHEL4b -]$ echo $var3 Svar4 
three four 

[paul@RHEL4b ~]$ bash 
[paul@RHEL4b ~]$ echo $var3 $var4 
four 


But it will not export to the parent shell (previous screenshot continued). 


paul@RHEL4b ~]$ export var5=five 
paul@RHEL4b -]$ echo $var3 Svar4 $var5 
four five 

paul@RHEL4b -]$ exit 

exit 

paul@RHEL4b ~]$ echo $var3 $var4 $var5 
three four 

paul@RHEL4b ~]$ 
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14.11. delineate variables 


Until now, we have seen that bash interprets a variable starting from a dollar sign, continuing 
until the first occurrence of a non-alphanumeric character that is not an underscore. In some 
situations, this can be a problem. This issue can be resolved with curly braces like in this 
example. 


[paul@RHEL4b ~]$ prefix-Super 

[paul@RHEL4b ~]$ echo Hello Sprefixman and $prefixgirl 
Hello and 

[paul@RHEL4b ~]$ echo Hello S${prefix}man and ${prefix}girl 
Hello Superman and Supergirl 

[paul@RHEL4b -]$ 


14.12. unbound variables 


The example below tries to display the value of the $MyVar variable, but it fails because the 
variable does not exist. By default the shell will display nothing when a variable is unbound 
(does not exist). 


[paul@RHELv4u3 gen]$ echo $MyVar 


[paul@RHELv4u3 gen]$ 


There is, however, the nounset shell option that you can use to generate an error when a 
variable does not exist. 


paul@laika:~$ set -u 
paul@laika:~$ echo $Myvar 
bash: Myvar: unbound variable 
paul@laika:~$ set +u 
paul@laika:~$ echo $Myvar 


paul@laika:~s 


In the bash shell set -u is identical to set -o nounset and likewise set +u is identical to set 
+0 nounset. 
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14.13. practice: shell variables 


]. Use echo to display Hello followed by your username. (use a bash variable!) 
2. Create a variable answer with a value of 42. 

3. Copy the value of $LANG to $MyLANG. 

4. List all current shell variables. 

5. List all exported shell variables. 

6. Do the env and set commands display your variable ? 

6. Destroy your answer variable. 

7. Create two variables, and export one of them. 

8. Display the exported variable in an interactive child shell. 


9. Create a variable, give it the value 'Dumb', create another variable with value 'do'. Use 
echo and the two variables to echo Dumbledore. 


10. Find the list of backslash escaped characters in the manual of bash. Add the time to your 
PS1 prompt. 
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14.14. solution: shell variables 


]. Use echo to display Hello followed by your username. (use a bash variable!) 


echo Hello SUSER 


2. Create a variable answer with a value of 42. 


answer-42 


3. Copy the value of $LANG to $MyLANG. 


MyLANG=$LANG 


4. List all current shell variables. 
set 


set |more on Ubuntu/Debian 


5. List all exported shell variables. 
env 


export 
declare -x 


6. Do the env and set commands display your variable ? 


env | more 
Set | more 


6. Destroy your answer variable. 


unset answer 


7. Create two variables, and export one of them. 


varl=1; export var2=2 


8. Display the exported variable in an interactive child shell. 


bash 
echo $var2 


9. Create a variable, give it the value 'Dumb', create another variable with value 'do'. Use 
echo and the two variables to echo Dumbledore. 

varx-Dumb; vary-do 

echo ${varx}leS{vary}re 


solution by Yves from Dexia : echo S$varx'le'$vary're' 
solution by Erwin from Telenet : echo "$varx"le"$vary"re 


10. Find the list of backslash escaped characters in the manual of bash. Add the time to your 
PS1 prompt. 


PS1-'Nt \u@\h \ws ' 
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options 


This chapter takes a brief look at child shells, embedded shells and shell options. 
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15.1. shell embedding 


Shells can be embedded on the command line, or in other words, the command line scan 
can spawn new processes containing a fork of the current shell. You can use variables to 
prove that new shells are created. In the screenshot below, the variable $varl only exists in 
the (temporary) sub shell. 


[paul@RHELv4u3 gen]$ echo $varl 
[paul@RHELv4u3 gen]$ echo $(varl=5;echo $varl) 
5 


[paul@RHELv4u3 gen]$ echo $varl 


[paul@RHELv4u3 gen]$ 
You can embed a shell in an embedded shell, this is called nested embedding of shells. 


This screenshot shows an embedded shell inside an embedded shell. 


paul@deb503:~$ A=shell 
paul@deb503:~$ echo SCSBSA $(B=sub;echo $CS$BS$A; echo $(C=sub;echo $CS$BSA) ) 
shell subshell subsubshell 


15.1.1. backticks 


Single embedding can be useful to avoid changing your current directory. The screenshot 
below uses backticks instead of dollar-bracket to embed. 


[paul@RHELv4u3 ~]$ echo 'cd /etc; ls -d * | grep pass” 
passwd passwd- passwd.OLD 
[paul@RHELv4u3 ~]$ 


You can only use the $() notation to nest embedded shells, backticks cannot do this. 
15.1.2. backticks or single quotes 


Placing the embedding between backticks uses one character less than the dollar and 
parenthesis combo. Be careful however, backticks are often confused with single quotes. 
The technical difference between ' and ` is significant! 


[paul@RHELv4u3 gen]$ echo ‘varl=5;echo $varl” 
5 

[paul@RHELv4u3 gen]$ echo 'varl=5;echo $varl' 
vari-5;echo Svari 

[paul@RHELv4u3 gen]$ 
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15.2. shell options 


Both set and unset are builtin shell commands. They can be used to set options of the bash 
shell itself. The next example will clarify this. By default, the shell will treat unset variables 
as a variable having no value. By setting the -u option, the shell will treat any reference to 
unset variables as an error. See the man page of bash for more information. 


paul@RHEL4b -]$ 


paul@RHEL4b ~]$ 
paul@RHEL4b -]$ 
paul@RHEL4b ~]$ 
paul@RHEL4b -]$ 


paul@RHEL4b -]$ 


echo $var123 


Ger =u 
echo $var123 


set +u 
echo $var123 


E 
E 
-bash: var123: unbound variable 
E 
E 


To list all the set options for your shell, use echo $-. The noclobber (or -C) option will be 
explained later in this book (in the I/O redirection chapter). 


paul@RHEL4b ~]$ 
himBH 
paul@RHEL4b ~]$ 
paul@RHEL4b ~]$ 
himuBCH 
paul@RHEL4b ~]$ 
paul@RHEL4b ~]$ 
himBH 
paul@RHEL4b ~]$ 


echo $- 


set -C ; set -u 
echo $- 


set +C ; set +u 
echo $- 


When typing set without options, you get a list of all variables without function when the 
shell is on posix mode. You can set bash in posix mode typing set -o posix. 
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15.3. practice: shell embedding 


1. Find the list of shell options in the man page of bash. What is the difference between set 
-u and set -o nounset? 


2. Activate nounset in your shell. Test that it shows an error message when using non- 
existing variables. 


3. Deactivate nounset. 
4. Execute cd /var and Is in an embedded shell. 


The echo command is only needed to show the result of the Is command. Omitting will result 
in the shell trying to execute the first file as a command. 


5. Create the variable embvar in an embedded shell and echo it. Does the variable exist in 
your current shell now ? 


6. Explain what "set -x" does. Can this be useful ? 


(optional)7. Given the following screenshot, add exactly four characters to that command 
line so that the total output is FirstMiddleLast. 


[paul@RHEL4b -]$ echo First; echo Middle; echo Last 


8. Display a long listing (Is -1) of the passwd command using the which command inside 
an embedded shell. 
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15.4. solution: shell embedding 


1. Find the list of shell options in the man page of bash. What is the difference between set 
-u and set -o nounset? 


read the manual of bash (man bash), search for nounset -- both mean the same thing. 


2. Activate nounset in your shell. Test that it shows an error message when using non- 
existing variables. 
set =u 


OR 
set. -o nounset 


Both these lines have the same effect. 


3. Deactivate nounset. 
set tu 


OR 
set to nounset 


4. Execute cd /var and ls in an embedded shell. 


echo $(cd /var ; ls) 


The echo command is only needed to show the result of the Is command. Omitting will result 
in the shell trying to execute the first file as a command. 


5. Create the variable embvar in an embedded shell and echo it. Does the variable exist in 
your current shell now ? 


echo $(embvar-emb;echo Sembvar) ; echo $embvar #the last echo fails 
Sembvar does not exist in your current shell 
6. Explain what "set -x" does. Can this be useful ? 


It displays shell expansion for troubleshooting your command. 


(optional)7. Given the following screenshot, add exactly four characters to that command 
line so that the total output is FirstMiddleLast. 


[paul@RHEL4b ~]$ echo First; echo Middle; echo Last 


echo -n First; echo -n Middle; echo Last 


8. Display a long listing (Is -1) of the passwd command using the which command inside 
an embedded shell. 


ls -1 $(which passwd) 
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Chapter 16. shell history 


The shell makes it easy for us to repeat commands, this chapter explains how. 
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16.1. repeating the last command 


To repeat the last command in bash, type !!. This is pronounced as bang bang. 


paul@debian5:~/test42$ echo this will be repeated > file42.txt 
paul@debian5:~/test42$ !! 

echo this will be repeated > file42.txt 

paulGdebian5:-/test42$ 


16.2. repeating other commands 


You can repeat other commands using one bang followed by one or more characters. The 
shell will repeat the last command that started with those characters. 


paul@debian5:~/test42$ touch file42 
paul@debian5:~/test42$ cat file42 
paulGdebian5:-/test42$ !to 

touch file42 

paul@debian5:~/test42$ 


16.3. history 


To see older commands, use history to display the shell command history (or use history 
n to see the last n commands). 


paul@debian5:~/test$ history 10 

38 mkdir test 

SOP Nec test 

40 touch filel 

41 echo hello > file2 

42 echo It is very cold today > winter.txt 


43 1s 

44 se 

45 cp winter.txt summer.txt 
ac ds =I 


47 history 10 
16.4. !n 


When typing ! followed by the number preceding the command you want repeated, then the 
shell will echo the command and execute it. 


paul@debian5:~/test$ !43 


ls 
filel file2 summer.txt  winter.txt 
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16.5. Ctrl-r 


Another option is to use ctrl-r to search in the history. In the screenshot below i only typed 
ctrl-r followed by four characters apti and it finds the last command containing these four 
consecutive characters. 


paul@debian5:~$ 
(reverse-i-search) apti': sudo aptitude install screen 


16.6. $HISTSIZE 


The $HISTSIZE variable determines the number of commands that will be remembered in 
your current environment. Most distributions default this variable to 500 or 1000. 


paul@debian5:~$ echo $HISTSIZE 
500 


You can change it to any value you like. 
paul@debian5:~$ HISTSIZE=15000 


paul@debian5:~$ echo $HISTSIZE 
15000 


16.7. $HISTFILE 


The SHISTFILE variable points to the file that contains your history. The bash shell defaults 
this value to -/.bash history. 


paul@debian5:~$ echo $HISTFILE 
/home/paul/.bash history 


A session history is saved to this file when you exit the session! 


Closing a gnome-terminal with the mouse, or typing reboot as root will NOT save your 
terminal's history. 


16.8. $HISTFILESIZE 


The number of commands kept in your history file can be set using $HISTFILESIZE. 


paul@debian5:~$ echo SHISTFILESIZE 
15000 
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16.9. prevent recording a command 


You can prevent a command from being recorded in history using a space prefix. 


paul@debian8:~/github$ echo abc 


abc 

paul@debian8:~/github$ echo def 
def 

paul@debian8:~/github$ echo ghi 
ghi 


paul@debian8:~/github$ history 3 
9501 echo abc 
9502 echo ghi 
9503" history 9 


16.10. (optional)regular expressions 


It is possible to use regular expressions when using the bang to repeat commands. The 
screenshot below switches 1 into 2. 


paul@debian5:~/test$ cat filel 
paul@debian5:~/test$ !c:s/1/2 
cat file2 

hello 

paul@debian5:~/test$ 


16.11. (optional) Korn shell history 


Repeating a command in the Korn shell is very similar. The Korn shell also has the history 
command, but uses the letter r to recall lines from history. 


This screenshot shows the history command. Note the different meaning of the parameter. 


$ history 17 
17 clear 

18 echo hoi 
LS) Misco) 12 
20 echo world 
21 history I7 


Repeating with r can be combined with the line numbers given by the history command, or 
with the first few letters of the command. 


$re 
echo world 
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16.12. practice: shell history 


1. Issue the command echo The answer to the meaning of life, the universe and 
everything is 42. 


2. Repeat the previous command using only two characters (there are two solutions!) 
3. Display the last 5 commands you typed. 


4. Issue the long echo from question 1 again, using the line numbers you received from the 
command in question 3. 


5. How many commands can be kept in memory for your current shell session ? 
6. Where are these commands stored when exiting the shell ? 


7. How many commands can be written to the history file when exiting your current shell 
session ? 


8. Make sure your current bash shell remembers the next 5000 commands you type. 


9. Open more than one console (by press Ctrl-shift-t in gnome-terminal, or by opening an 
extra putty.exe in MS Windows) with the same user account. When is command history 
written to the history file ? 
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16.13. solution: shell history 


1. Issue the command echo The answer to the meaning of life, the universe and 
everything is 42. 


echo The answer to the meaning of life, the universe and everything is 42 


2. Repeat the previous command using only two characters (there are two solutions!) 


1] 
OR 
le 


3. Display the last 5 commands you typed. 
paulfubul010:-$ history 5 

52 ITs =L 

53s 

54 rdf =h | grep sda 


55 echo The answer to the meaning of life, the universe and everything is 42 
56 history 5 


You will receive different line numbers. 


4. Issue the long echo from question 1 again, using the line numbers you received from the 
command in question 3. 


paul@ubu1010:~$ 155 


echo The answer to the meaning of life, the universe and everything is 42 
The answer to the meaning of life, the universe and everything is 42 


5. How many commands can be kept in memory for your current shell session ? 


echo SHISTSIZE 


6. Where are these commands stored when exiting the shell ? 


echo SHISTFILE 


7. How many commands can be written to the history file when exiting your current shell 
session ? 


echo SHISTFILESIZE 


8. Make sure your current bash shell remembers the next 5000 commands you type. 


HISTSIZE-5000 


9. Open more than one console (by press Ctrl-shift-t in gnome-terminal, or by opening an 
extra putty.exe in MS Windows) with the same user account. When is command history 
written to the history file ? 


when you type exit 
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Chapter 17. file globbing 


The shell is also responsible for file globbing (or dynamic filename generation). This chapter 
will explain file globbing. 
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17.1. * asterisk 


The asterisk * is interpreted by the shell as a sign to generate filenames, matching the asterisk 
to any combination of characters (even none). When no path is given, the shell will use 
filenames in the current directory. See the man page of glob(7) for more information. (This 
is part of LPI topic 1.103.3.) 


paul@RHELv4u3 gen]$ 1s 

filel file2 file3 File4 File55 FileA fileab Fileab FileAB fileabc 
paul@RHELv4u3 gen]$ ls File* 

File4 File55 FileA Fileab FileAB 
paul@RHELv4u3 gen]$ 1s file* 

filel file2 file3 fileab fileabc 
paul@RHELv4u3 gen]$ ls *ile55 
File55 

paul@RHELv4u3 gen]$ ls F*ile55 
File55 

paul@RHELv4u3 gen]$ ls F*55 

File55 


paul@RHELv4u3 gen]$ 


17.2. ? question mark 


Similar to the asterisk, the question mark ? is interpreted by the shell as a sign to generate 
filenames, matching the question mark with exactly one character. 


paul@RHELv4u3 gen]$ 1s 

filel file2 file3 File4 File55 FileA fileab Fileab FileAB fileabc 
paul@RHELv4u3 gen]$ ls File? 

File4 FileA 

paul@RHELv4u3 gen]$ ls Fil?4 

File4 

paul@RHELv4u3 gen]$ 1s Fil?? 


ileab FileAB 


E 
hb 
paul@RHELv4u3 gen]$ ls File?? 
E 
E 


paul@RH 
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17.3. [] square brackets 


The square bracket [ is interpreted by the shell as a sign to generate filenames, matching 
any of the characters between [ and the first subsequent ]. The order in this list between the 
brackets is not important. Each pair of brackets is replaced by exactly one character. 


File55 FileA fileab Fileab FileAB fileabc 


paul@RHELv4u3 gen]$ 1s 

filel file2 file3 File4 
paul@RHELv4u3 gen]$ 1s File[5A 
FileA 

paul@RHELv4u3 gen]$ 1s File[A5 
FileA 

paul@RHELv4u3 gen]$ ls File[A5 
File55 

paul@RHELv4u3 gen]$ ls File[a5 
File55 Fileab 

paul@RHELv4u3 gen]$ 1s File[a5 
ls: File[a5][5b] [abcdefghijklm]: 
paul@RHELv4u3 gen]$ ls file[a5 
fileabc 

paul@RHELv4u3 gen]$ 


[5b] 
[5b] 
[5b] [abcdefghijklm] 


No such file or directory 
[5b] [abcdefghijklm] 


You can also exclude characters from a list between square brackets with the exclamation 
mark !. And you are allowed to make combinations of these wild cards. 


paul@RHELv4u3 gen]$ 
filel file2 file3 
paul@RHELv4u3 gen]$ 
fileab 
paul@RHELv4u3 gen]$ 
filel file2 file3 
paul@RHELv4u3 gen]$ 
fileab 
paul@RHELv4u3 gen]$ 


ls 
File4 


File55 FileA fileab Fileab FileAB fileabc 
ls file[a5][!2] 


Ts EE CST oar 
fileabc 
Is eave quise 


fileab 
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17.4. a-z and 0-9 ranges 


The bash shell will also understand ranges of characters between brackets. 


paul@RHELv4u3 gen]$ 1s 
filel file3 File55 fileab FileAB fileabc 
file2 File4 FileA Fileab fileab2 
paul@RHELv4u3 gen]$ ls file[a-z]* 
fileab fileab2 fileabc 
paul@RHELv4u3 gen]$ ls file[0-9] 
filel file2 file3 
E 


Lv4u3 gen]$ ls file[a-z][a-z][0-9]* 


paul@RHELv4u3 gen]$ 


17.5. $LANG and square brackets 


But, don't forget the influence of the LANG variable. Some languages include lower case 
letters in an upper case range (and vice versa). 


paul@RHELv4u4:~/test$ ls [A-Z]ile? 
filel file2 file3 File4 
paul@RHELv4u4:~/test$ ls [a-z]ile? 
filel file2 file3 File4 
paul@RHELv4u4:~/test$ echo SLANG 
en_US.UTF-8 

paul@RHELv4u4:~/test$ LANG=C 
paul@RHELv4u4:~/test$ echo SLANG 


paul@RHELv4u4:~/test$ ls [a-z]ile? 
filel file2 file3 
paul@RHELv4u4:~/test$ ls [A-Z]ile? 


paul@RHELv4u4:~/test$ 


If $LC_ALL is set, then this will also need to be reset to prevent file globbing. 
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17.6. preventing file globbing 


The screenshot below should be no surprise. The echo * will echo a * when in an empty 
directory. And it will echo the names of all files when the directory is not empty. 


paulfubul010:-$ mkdir test42 
paulQubul010:-$ cd test42 
paul@ubu1010:~/test42$ echo * 

* 

paul@ubul010:~/test42$ touch file42 file33 
paul@ubul010:~/test42$ echo * 

file33 file42 


Globbing can be prevented using quotes or by escaping the special characters, as shown in 
this screenshot. 


paul@ubu1010:~/test42$ echo * 
file33 file42 
paul@ubu1010:~/test42$ echo \* 


* 


paul@ubu1010:~/test42$ echo '*' 


* 


paul@ubu1010:~/test42$ echo "*" 


* 
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17.7. practice: shell globbing 


]. Create a test directory and enter it. 


2. Create the following files : 


filel 
filelO 
fmt 
file2 
File2 
File3 
file33 
fileAB 
filea 
fileA 
fileAAA 
file( 
milen? 


(the last one has 6 characters including a space) 

3. List (with Is) all files starting with file 

4. List (with Is) all files starting with File 

5. List (with Is) all files starting with file and ending in a number. 

6. List (with Is) all files starting with file and ending with a letter 

7. List (with Is) all files starting with File and having a digit as fifth character. 


8. List (with Is) all files starting with File and having a digit as fifth character and nothing 
else. 


9. List (with Is) all files starting with a letter and ending in a number. 

10. List (with Is) all files that have exactly five characters. 

11. List (with Is) all files that start with f or F and end with 3 or A. 

12. List (with Is) all files that start with f have i or R as second character and end in a number. 
13. List all files that do not start with the letter F. 

14. Copy the value of $LANG to $MyLANG. 

15. Show the influence of SLANG in listing A-Z or a-z ranges. 


16. You receive information that one of your servers was cracked, the cracker probably 
replaced the Is command. You know that the echo command is safe to use. Can echo replace 
Is ? How can you list the files in the current directory with echo ? 


17. Is there another command besides cd to change directories ? 
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17.8. solution: shell globbing 


1. Create a test directory and enter it. 


mkdir testdir; cd testdir 


2. Create the following files : 


fte 
filelO 
fileli 
mez 
File2 
File3 
file33 
fileAB 
filea 
fileA 
fileAAA 
file ( 
file 2 


(the last one has 6 characters including a space) 
touch filel filel0 filell file2 File2 File3 
touch file33 fileAB filea fileA fileAAA 


touche ase 
Eouchs ES Pez 


3. List (with Is) all files starting with file 


ls file* 


4. List (with Is) all files starting with File 


ls File* 

5. List (with Is) all files starting with file and ending in a number. 
ls file*[0-9] 

6. List (with Is) all files starting with file and ending with a letter 
ls file*[a-z] 


7. List (with Is) all files starting with File and having a digit as fifth character. 


Ts Erler] 


8. List (with Is) all files starting with File and having a digit as fifth character and nothing 
else. 


T Spas 


9. List (with Is) all files starting with a letter and ending in a number. 


ts [era | T 0=9] 


10. List (with Is) all files that have exactly five characters. 
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le 229292 


11. List (with Is) all files that start with f or F and end with 3 or A. 

ls [fF]*[3A] 

12. List (with Is) all files that start with f have i or R as second character and end in a number. 
ls f[iR]*[0-9] 

13. List all files that do not start with the letter F. 

ls [!F]* 

14. Copy the value of $LANG to $MyLANG. 


My LANG=$ LANG 


15. Show the influence of $LANG in listing A-Z or a-z ranges. 


see example in book 


16. You receive information that one of your servers was cracked, the cracker probably 
replaced the Is command. You know that the echo command is safe to use. Can echo replace 
Is ? How can you list the files in the current directory with echo ? 


echo * 


17. Is there another command besides cd to change directories ? 


pushd popd 
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Chapter 18. I/O redirection 


One of the powers of the Unix command line is the use of input/output redirection and 
pipes. 


This chapter explains redirection of input, output and error streams. 
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18.1. stdin, stdout, and stderr 


The bash shell has three basic streams; it takes input from stdin (stream 0), it sends output 
to stdout (stream 1) and it sends error messages to stderr (stream 2) . 


The drawing below has a graphical interpretation of these three streams. 


The keyboard often serves as stdin, whereas stdout and stderr both go to the display. This 
can be confusing to new Linux users because there is no obvious way to recognize stdout 
from stderr. Experienced users know that separating output from errors can be very useful. 


dis 
—> d | je — € 


— isplay 
The next sections will explain how to redirect these streams. 
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18.2. output redirection 
18.2.1. » stdout 


stdout can be redirected with a greater than sign. While scanning the line, the shell will 
see the > sign and will clear the file. 


> file 


> 


display 


keyboard 


> 


The > notation is in fact the abbreviation of 1> (stdout being referred to as stream 1). 


[paul@RHELv4u3 -]$ echo It is cold today! 

Te is cold today! 

[paul@RHELv4u3 ~]$ echo It is cold today! > winter.txt 
[paul@RHELv4u3 ~]$ cat winter.txt 

reais cola today! 

[paul@RHELv4u3 ~]$ 


Note that the bash shell effectively removes the redirection from the command line before 
argument 0 is executed. This means that in the case of this command: 


echo hello > greetings.txt 


the shell only counts two arguments (echo = argument 0, hello = argument 1). The redirection 
is removed before the argument counting takes place. 


18.2.2. output file is erased 


While scanning the line, the shell will see the > sign and will clear the file! Since this 
happens before resolving argument 0, this means that even when the command fails, the 
file will have been cleared! 


[paul@RHELv4u3 ~]$ cat winter.txt 

Te iS cold. today! 

[paul@RHELv4u3 ~]$ zcho It is cold today! > winter.txt 
-bash: zcho: command not found 

[paul@RHELv4u3 ~]$ cat winter.txt 

[paul@RHELv4u3 ~]$ 
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18.2.3. noclobber 


Erasing a file while using > can be prevented by setting the noclobber option. 


[paul@RHELv4u3 ~]$ cat winter.txt 

It is cold today! 

[paul@RHELv4u3 -]$ set -o noclobber 

[paul@RHELv4u3 ~]$ echo It is cold today! > winter.txt 
-bash: winter.txt: cannot overwrite existing file 
[paul@RHELv4u3 -]$ set +o noclobber 

[paul@RHELv4u3 ~]$ 


18.2.4. overruling noclobber 


The noclobber can be overruled with >|. 


[paul@RHELv4u3 -]$ set -o noclobber 

[paul@RHELv4u3 ~]$ echo It is cold today! > winter.txt 
-bash: winter.txt: cannot overwrite existing file 
[paul@RHELv4u3 -]$ echo It is very cold today! >| winter.txt 
[paul@RHELv4u3 ~]$ cat winter.txt 

It is very cold today! 

[paul@RHELv4u3 -]$ 


18.2.5. >> append 


Use >> to append output to a file. 


paul@RHELv4u3 ~]$ echo It is cold today! > winter.txt 
paul@RHELv4u3 ~]$ cat winter.txt 

It is cold today! 

paul@RHELv4u3 ~]$ echo Where is the summer ? >> winter.txt 
paul@RHELv4u3 ~]$ cat winter.txt 

Lt u$s cold today! 

Where is the summer ? 

paul@RHELv4u3 -]$ 
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18.3. error redirection 
18.3.1. 2» stderr 


Redirecting stderr is done with 2». This can be very useful to prevent error messages from 


cluttering your screen. 
d splay . 


2»file 


keyboard 


e 


The screenshot below shows redirection of stdout to a file, and stderr to /dev/null. Writing 
1> is the same as >. 


[paul@RHELv4u3 -]$ find / > allfiles.txt 2» /dev/null 
[paul@RHELv4u3 -]$ 


18.3.2. 22 &1 


To redirect both stdout and stderr to the same file, use 2>4 1. 


[paul@RHELv4u3 -]$ find / > allfiles and errors.txt 2>41 
[paul@RHELv4u3 ~]$ 


Note that the order of redirections is significant. For example, the command 


ls > dirlist 2>&1 


directs both standard output (file descriptor 1) and standard error (file descriptor 2) to the 
file dirlist, while the command 


Is 2561 > dīirlist 


directs only the standard output to file dirlist, because the standard error made a copy of the 
standard output before the standard output was redirected to dirlist. 
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18.4. output redirection and pipes 


By default you cannot grep inside stderr when using pipes on the command line, because 
only stdout is passed. 


paul@debian7:~$ rm file42 file33 filel201 | grep file42 
rm: cannot remove ‘file42’: No such file or directory 
rm: cannot remove ‘file33’: No such file or directory 
rm: cannot remove "file1201': No such file or directory 


With 2» &1 you can force stderr to go to stdout. This enables the next command in the 
pipe to act on both streams. 


paulGdebian7:-$ rm file42 file33 filel201 2>&1 | grep file42 
rm: cannot remove ‘file42’: No such file or directory 


You cannot use both 1» &2 and 2» &1 to switch stdout and stderr. 


paul@debian7:~$ rm file42 file33 filel1201 2»&1 1>&2 | grep file42 
rm: cannot remove "file42': No such file or directory 
paul@debian7:~$ echo file42 2>&1 1>&2 | sed 's/file42/FILE42/' 
FILE42 


You need a third stream to switch stdout and stderr after a pipe symbol. 


paul@debian7:~$ echo file42 3>&1 1>&2 2>&3 | sed 's/file42/FILE42/' 
file42 

paul@debian7:~$ rm file42 3>&1 1>&2 2>&3 | sed 's/file42/FILE42/' 
rm: cannot remove ‘FILE42’: No such file or directory 


18.5. joining stdout and stderr 


The &> construction will put both stdout and stderr in one stream (to a file). 


paul@debian7:~S rm file42 &» out and err 
paul@debian7:~$ cat out and err 

rm: cannot remove ‘file42’: No such file or directory 
paul@debian7:~$ echo file42 &» out and err 
paul@debian7:~$ cat out and err 

file42 

paul@debian7:~S$ 


176 


T/O redirection 


18.6. input redirection 
18.6.1. « stdin 


Redirecting stdin is done with « (short for 0<). 


[paul@RHEL4b -]$ cat « text.txt 

one 

two 

[paul@RHEL4b -]$ tr 'onetw' 'ONEZZ' « text.txt 
ONE 
ZZO 

[paul@RHEL4b -]$ 


18.6.2. << here document 


The here document (sometimes called here-is-document) is a way to append input until a 
certain sequence (usually EOF) is encountered. The EOF marker can be typed literally or 
can be called with Ctrl-D. 


[paul@RHEL4b ~] $ cat <<EOF > text.txt 
> one 

> two 

> EOF 

paul@RHEL4b -]$ cat text.txt 


paul@RHEL4b -]$ cat <<brol > text. txt 


paul@RHEL4b -]$ cat text.txt 


paul@RHEL4b ~]$ 


18.6.3. <<< here string 
The here string can be used to directly pass strings to a command. The result is the same 
as using echo string | command (but you have one less process running). 
paul@ubu1l110~S base64 <<< linux-training.be 
bGludXgtdHJhaW5pbmcuYmUK 


paulfubul110-$ base64 -d <<< bGludXgtdHJhaW5pbmcuYmUK 
linux-training.be 


See rfc 3548 for more information about base64. 
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18.7. confusing redirection 


The shell will scan the whole line before applying redirection. The following command line 
is very readable and is correct. 


cat winter.txt » snow.txt 2» errors.txt 
But this one is also correct, but less readable. 


2» errors.txt cat winter.txt » snow.txt 


Even this will be understood perfectly by the shell. 


« winter.txt > snow.txt 2» errors.txt cat 


18.8. quick file clear 


So what is the quickest way to clear a file ? 


>foo 


And what is the quickest way to clear a file when the noclobber option is set ? 


>|bar 
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18.9. practice: input/output redirection 


1. Activate the noclobber shell option. 

2. Verify that noclobber is active by repeating an Is on /etc/ with redirected output to a file. 
3. When listing all shell options, which character represents the noclobber option ? 

4. Deactivate the noclobber option. 


5. Make sure you have two shells open on the same computer. Create an empty tailing.txt 
file. Then type tail -f tailing.txt. Use the second shell to append a line of text to that file. 
Verify that the first shell displays this line. 


6. Create a file that contains the names of five people. Use cat and output redirection to 
create the file and use a here document to end the input. 
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18.10. solution: input/output redirection 


1. Activate the noclobber shell option. 


set -o noclobber 
set -C 


2. Verify that noclobber is active by repeating an Is on /etc/ with redirected output to a file. 


ls cue > Sue pide 
ls /etc > etc.txt (should not work) 


4. When listing all shell options, which character represents the noclobber option ? 


echo $- (noclobber is visible as C) 


5. Deactivate the noclobber option. 


set +o noclobber 


6. Make sure you have two shells open on the same computer. Create an empty tailing.txt 
file. Then type tail -f tailing.txt. Use the second shell to append a line of text to that file. 
Verify that the first shell displays this line. 


paul8deb503:-$ > tailing.txt 
paul@deb503:~S tail -f tailing.txt 
hello 

world 


in the other shell: 
paul@deb503:~S$ echo hello >> tailing.txt 
paul@deb503:~$ echo world >> tailing.txt 


7. Create a file that contains the names of five people. Use cat and output redirection to 
create the file and use a here document to end the input. 


paul@deb503:~$ cat > tennis.txt << ace 
Justine Henin 

Venus Williams 

Serena Williams 

Martina Hingis 

Kim Clijsters 

> ace 

paul@deb503:~S cat tennis.txt 
Justine Henin 

Venus Williams 

Serena Williams 

Martina Hingis 

Kim Clijsters 

paul@deb503:~$ 


VEEN SEN SNE: 
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Commands that are created to be used with a pipe are often called filters. These filters 
are very small programs that do one specific thing very efficiently. They can be used as 
building blocks. 


This chapter will introduce you to the most common filters. The combination of simple 
commands and filters in a long pipe allows you to design elegant solutions. 
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19.1. cat 


When between two pipes, the cat command does nothing (except putting stdin on stdout). 


[paul@RHEL4b pipes]$ tac count.txt | cat | cat | cat | cat | cat 
mie 

four 

three 

two 

one 

[paul@RHEL4b pipes]$ 


19.2. tee 


Writing long pipes in Unix is fun, but sometimes you may want intermediate results. This 
is were tee comes in handy. The tee filter puts stdin on stdout and also into a file. So tee is 
almost the same as cat, except that it has two identical outputs. 


[paul@RHEL4b pipes]$ tac count.txt | tee temp.txt | tac 
one 

two 

three 

FOUr 

five 

[paul@RHEL4b pipes]$ cat temp.txt 
five 

four 

three 

two 

one 

[paul@RHEL4b pipes]$ 


19.3. grep 


The grep filter is famous among Unix users. The most common use of grep is to filter lines 
of text containing (or not containing) a certain string. 


[paul@RHEL4b pipes]$ cat tennis.txt 

Amelie Mauresmo, Fra 

Kim Clijsters, BEL 

Justine Henin, Bel 

Serena Williams, usa 

Venus Williams, USA 

[paul@RHEL4b pipes]$ cat tennis.txt grep Williams 
Serena Williams, usa 

Venus Williams, USA 


You can write this without the cat. 
[paul@RHEL4b pipes]$ grep Williams tennis.txt 


Serena Williams, usa 
Venus Williams, USA 


One of the most useful options of grep is grep -i which filters in a case insensitive way. 
[paul@RHEL4b pipes]$ grep Bel tennis.txt 


Justine Henin, Bel 
[paul@RHEL4b pipes]$ grep -i Bel tennis.txt 
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Kim Clijsters, BEL 
Justine Henin, Bel 
[paul@RHEL4b pipes]$ 


Another very useful option is grep -v which outputs lines not matching the string. 


[paul@RHEL4b pipes]$ grep -v Fra tennis.txt 
Kim Clijsters, BEL 

Justine Henin, Bel 

Serena Williams, usa 

Venus Williams, USA 

[paul@RHEL4b pipes]$ 


And of course, both options can be combined to filter all lines not containing a case 
insensitive string. 


[paul@RHEL4b pipes]$ grep -vi usa tennis.txt 
Amelie Mauresmo, Fra 

Kim Clijsters, BEL 

Justine Henin, Bel 

[paul@RHEL4b pipes]$ 


With grep -A1 one line after the result is also displayed. 


paul@debian5:~/pipes$ grep -Al Henin tennis.txt 
Justine Henin, Bel 
Serena Williams, usa 


With grep -B1 one line before the result is also displayed. 


paul@debian5:~/pipes$ grep -B1 Henin tennis.txt 
Kim Clijsters, BEL 
Justine Henin, Bel 


With grep -C1 (context) one line before and one after are also displayed. All three options 
(A,B, and C) can display any number of lines (using e.g. A2, B4 or C20). 


paul@debian5:~/pipes$ grep -C1 Henin tennis.txt 
Kim Clijsters, BEL 

Justine Henin, Bel 

Serena Williams, usa 
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19.4. cut 


The cut filter can select columns from files, depending on a delimiter or a count of bytes. 
The screenshot below uses cut to filter for the username and userid in the /etc/passwd file. 
It uses the colon as a delimiter, and selects fields 1 and 3. 


[[paulGRHEL4b pipes]$ cut -d: -f1,3 /etc/passwd | tail -4 
Eoo 

Pfaff:511 

Heim vaca 5116 

Hermione:517 

[paul@RHEL4b pipes]$ 


When using a space as the delimiter for cut, you have to quote the space. 


[paul@RHEL4b pipes]$ cut -d" " -fl tennis.txt 
Amelie 

Kim 

Justine 

Serena 

Venus 

[paul@RHEL4b pipes]$ 


This example uses cut to display the second to the seventh character of /etc/passwd. 


[paul@RHEL4b pipes]$ cut -c2-7 /etc/passwd | tail -4 
HIE ORs <a 

farf 

arry:x 

ermion 

[paul@RHEL4b pipes]$ 


19.5. tr 


You can translate characters with tr. The screenshot shows the translation of all occurrences 
of e to E. 


E 


[paul@RHEL4b pipes]$ cat tennis.txt | te Nero! 
AmE1iE MaurEsmo, Fra 

Kim ClijstErs, BEL 

JustinE HEnin, BEI 

SErEna Williams, usa 

VEnus Williams, USA 


Here we set all letters to uppercase by defining two ranges. 


[paul@RHEL4b pipes]$ cat tennis.txt | EE Saale MAZIM 
AMELIE MAURESMO, FRA 

KIM CLIJSTERS, BEL 

JUSTINE HENIN, BEL 

SERENA WILLIAMS, USA 

VENUS WILLIAMS, USA 

[paul@RHEL4b pipes]$ 


Here we translate all newlines to spaces. 
[paul@RHEL4b pipes]$ cat count.txt 


one 
two 
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three 

four 

five 

[paul@RHEL4b pipes]$ cat count.txt | [Eam 
one two three four five [paul@RHEL4b pipes]$ 


The tr -s filter can also be used to squeeze multiple occurrences of a character to one. 


[paul@RHEL4b pipes]$ cat spaces.txt 
one two three 
four five six 
[paul@RHEL4b pipes]$ cat spaces.txt Eee c 
one two three 
four five six 
[paul@RHEL4b pipes]$ 


You can also use tr to 'encrypt' texts with rot13. 


[paul@RHEL4b pipes]$ cat count.txt | tr 'a-z' 'nopqrstuvwxyzabcdefghijklm' 
bar 

gjb 

guerr 

sbhe 

habe 

[paul@RHEL4b pipes]$ cat count.txt tr Vaz" In=za=-m! 
bar 

gjb 

guerr 

sbhe 

svir 

[paul@RHEL4b pipes]$ 


This last example uses tr -d to delete characters. 


paul@debian5:~/pipes$ cat tennis.txt | tr -de 
Amli Maursmo, Fra 

Emu Gata Sites) BEL 

Justin Hnin, Bl 

Srna Williams, usa 

Vnus Williams, USA 


19.6. wc 


Counting words, lines and characters is easy with wc. 


paul@RHEL4b pipes]$ wc tennis.txt 

5 15 100 tennis.txt 
paul@RHEL4b pipes]$ wc -1 tennis.txt 
5 tennis.txt 


paul@RHEL4b pipes]$ wc -w tennis.txt 
15 tennis.txt 

paul@RHEL4b pipes]$ wc -c tennis.txt 
100 tennis.txt 

paul@RHEL4b pipes]$ 
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19.7. sort 


The sort filter will default to an alphabetical sort. 


paul@debian5:~/pipes$ cat music.txt 
Queen 

Brel 
Led Zeppelin 
Abba 
paul@debian5:~/pipes$ sort music.txt 
Abba 
Brel 
Led Zeppelin 
Queen 


But the sort filter has many options to tweak its usage. This example shows sorting different 
columns (column 1 or column 2). 


[paul@RHEL4b pipes]$ sort -k1 country.txt 
Belgium, Brussels, 10 

France, Paris, 60 

Germany, Berlin, 100 

Iran, Teheran, 70 

Italy, Rome, 50 

[paul@RHEL4b pipes]$ sort -k2 country.txt 
Germany, Berlin, 100 

Belgium, Brussels, 10 

France, Paris, 60 

Italy, Rome, 50 

Iran, Teheran, 70 


The screenshot below shows the difference between an alphabetical sort and a numerical 
sort (both on the third column). 


[paul@RHEL4b pipes]$ sort -k3 country.txt 
Belgium, Brussels, 10 

Germany, Berlin, 100 

Italy, Rome, 50 

France, Paris, 60 

Iran, Teheran, 70 

[paul@RHEL4b pipes]$ sort -n -k3 country.txt 
Belgium, Brussels, 10 

Italy, Rome, 50 

France, Paris, 60 

Iran, Teheran, 70 

Germany, Berlin, 100 
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19.8. uniq 


With uniq you can remove duplicates from a sorted list. 


paul@debian5:~/pipes$ cat music.txt 
Queen 

Brel 
Queen 
Abba 
paul@debian5:~/pipes$ sort music.txt 
Abba 
Brel 
Queen 

Queen 

paul@debian5:~/pipes$ sort music.txt |uniq 
Abba 
Brel 
Queen 


uniq can also count occurrences with the -c option. 


paul@debian5:~/pipes$ sort music.txt | uniq E 
1 Abba 
1 Brel 
2 Queen 
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19.9. comm 


Comparing streams (or files) can be done with the comm. By default comm will output 
three columns. In this example, Abba, Cure and Queen are in both lists, Bowie and Sweet 
are only in the first file, Turner is only in the second. 


paulGdebian5:-/pipes$ cat > listl.txt 
Abba 
Bowie 
Cure 
Queen 
Sweet 
paulGdebian5:-/pipes$ cat > list2.txt 
Abba 
Cure 
Queen 
Turner 
paulGdebian5:-/pipes$ comm listl.txt list2.txt 
Abba 
Bowie 
Cure 
Queen 
Sweet 
Turner 


The output of comm can be easier to read when outputting only a single column. The digits 
point out which output columns should not be displayed. 


paul@debian5:~/pipes$ comm -12 listl.txt list2.txt 
Abba 

Cure 

Queen 

paul@debrans<~/pipess comm -13 listi txt Mist2stxt 
Turner 

paul@debian5:~/pipes$ comm -23 listl.txt list2.txt 
Bowie 

Sweet 
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19.10. od 


European humans like to work with ascii characters, but computers store files in bytes. The 
example below creates a simple file, and then uses od to show the contents of the file in 
hexadecimal bytes 


paul@laika:~/test$ cat > text.txt 

abcdefg 

1234567 

paul@laika:~/test$ od -t xl text.txt 

0000000001261 62 63°64 65 66 67 Oa 31 32 353 34 35 36 37 Oa 
0000020 


The same file can also be displayed in octal bytes. 


paul@laika:~/test$ od -b text.txt 
0000000 141 142 143 144 145 146 147 012 061 062 063 064 065 066 067 012 
0000020 


And here is the file in ascii (or backslashed) characters. 


paul@laika:~/test$ od -c text.txt 
0000000 a b (e d e E g \n 1 2 3 4 5 6 UND 
0000020 
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19.11. sed 


The stream editor sed can perform editing functions in the stream, using regular 
expressions. 


paul@debian5:~/pipes$ echo level5 | sed 's/5/42/' 
level42 

paul@debian5:~/pipes$ echo level5 | sed 's/level/jump/' 
jump5 


Add g for global replacements (all occurrences of the string per line). 


paul@debian5:~/pipes$ echo level5 level7 | sed 's/level/jump/' 
jump5 level7 
paul@debian5:~/pipes$ echo level5 level7 | sed 's/level/jump/g' 
jump5 jump7 


With d you can remove lines from a stream containing a character. 


paul@debian5:~/test42$ cat tennis.txt 
Venus Williams, USA 

Martina Hingis, SUI 

Justine Henin, BE 

Serena williams, USA 

Kim Clijsters, BE 

Yanina Wickmayer, BE 
paul@debian5:~/test42$ cat tennis.txt | sed '/BE/d' 
Venus Williams, USA 

Martina Hingis, SUI 

Serena williams, USA 
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19.12. pipe examples 
19.12.1. who | wc 


How many users are logged on to this system ? 


[paul@RHEL4b pipes]$ who 


root Biel Jul 25 T050 

paul pts/0 Jul 25 09:29 (laika) 
Harry pts/1 nou eis 2:26 (BaCry) 
paul pts/2 Jul 25 12:26 (pasha) 
[paul@RHEL4b pipes]$ who | wc -1 

4 


19.12.2. who | cut | sort 


Display a sorted list of logged on users. 


[paul@RHEL4b pipes]$ who | GNE Sel Y cea sort 
Harry 

paul 

paul 

root 


Display a sorted list of logged on users, but every user only once . 


[paul@RHEL4b pipes]$ who | ems —el Y Sail | sort | uniq 
Harry 

paul 

root 


19.12.3. grep | cut 


Display a list of all bash user accounts on this computer. Users accounts are explained in 
detail later. 


paul@debian5:~$ grep bash /etc/passwd 
goose “Oot. / bin/bash 
paul:x:1000:1000:paul,,,:/home/paul:/bin/bash 
serena:x:1001:1001::/home/serena:/bin/bash 
paul@debian5:~$ grep bash /etc/passwd | Cle Sele se il 
root 

paul 

serena 
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19.13. practice: filters 


]. Put a sorted list of all bash users in bashusers.txt. 
2. Put a sorted list of all logged on users in onlineusers.txt. 
3. Make a list of all filenames in /ete that contain the string conf in their filename. 


4. Make a sorted list of all files in /etc that contain the case insensitive string conf in their 
filename. 


5. Look at the output of /sbin/ifconfig. Write a line that displays only ip address and the 
subnet mask. 


6. Write a line that removes all non-letters from a stream. 
7. Write a line that receives a text file, and outputs all words on a separate line. 


8. Write a spell checker on the command line. (There may be a dictionary in /usr/share/ 
dict/ .) 
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19.14. solution: filters 


]. Put a sorted list of all bash users in bashusers.txt. 


grep bash /etc/passwd | cut -d: -f1 | sort > bashusers.txt 


2. Put a sorted list of all logged on users in onlineusers.txt. 


who | cut -d' ' -f1 | sort > onlineusers.txt 


3. Make a list of all filenames in /ete that contain the string conf in their filename. 


ls /etc | grep conf 


4. Make a sorted list of all files in /etc that contain the case insensitive string conf in their 
filename. 


ls /etc | grep -i conf | sort 


5. Look at the output of /sbin/ifconfig. Write a line that displays only ip address and the 
subnet mask. 


AAA | head 2 | gzep aet SI | a ES 


6. Write a line that removes all non-letters from a stream. 


paul@deb503:~$ cat text 

This is, yes really! , a text with ?&* too many strSange# characters ;-) 
paul@deb503:~S cat text tre ad specs ay 

This is yes really a text with too many strange characters 


7. Write a line that receives a text file, and outputs all words on a separate line. 


paul@deb503:~S cat text2 
it is very cold today without the sun 


paul@deb503:~$ cat text2 | tr ' ' 'An' 
aste 

Bb 

very 

cold 

today 

without 

the 

sun 


8. Write a spell checker on the command line. (There may be a dictionary in /usr/share/ 
dict/ .) 


paul@rhel ~$ echo "The zun is shining today" > text 


paul@rhel ~$ cat > DICT 
is 

shining 

sun 

the 
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today 


paul@rhel ~$ cat text ER UA. A TENES TNI | sort uniq comm =23 — DICT 
zun 


You could also add the solution from question number 6 to remove non-letters, and tr -s ' 
' to remove redundant spaces. 
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Chapter 20. basic Unix tools 


This chapter introduces commands to find or locate files and to compress files, together 
with other common tools that were not discussed before. While the tools discussed here are 
technically not considered filters, they can be used in pipes. 
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20.1. find 


The find command can be very useful at the start of a pipe to search for files. Here are some 
examples. You might want to add 2»/dev/null to the command lines to avoid cluttering your 
screen with error messages. 


Find all files in /etc and put the list in etcfiles.txt 


find Vete > etcfiles.txt 


Find all files of the entire system and put the list in allfiles.txt 


find / > alllfiles.txt 


Find files that end in .conf in the current directory (and all subdirs). 


faind y nane o 


Find files of type file (not directory, pipe or etc.) that end in .conf. 


tid. type E Nane e conf" 


Find files of type directory that end in .bak . 


find /data -type d -name "*.bak" 


Find files that are newer than file42.txt 


find . -newer file42.txt 


Find can also execute another command on every file found. This example will look for 
* odf files and copy them to /backup/. 


find /data name =. odf “exec cp {} /backup/ Y; 
Find can also execute, after your confirmation, another command on every file found. This 


example will remove *.odf files if you approve of it for every file found. 


find /data -name "*.odf" -ok rm {} V; 
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20.2. locate 


The locate tool is very different from find in that it uses an index to locate files. This is a 
lot faster than traversing all the directories, but it also means that it is always outdated. If 
the index does not exist yet, then you have to create it (as root on Red Hat Enterprise Linux) 
with the updatedb command. 


[paul@RHEL4b -]$ locate Samba 

warning: locate: could not open database: /var/lib/slocate/slocate.db:... 
warning: You need to run the 'updatedb' command (as root) to create th... 
Please have a look at /etc/updatedb.conf to enable the daily cron job. 
[paul@RHEL4b -]$ updatedb 

fatal error: updatedb: You are not authorized to create a default sloc... 
[paul@RHEL4b -]$ su - 

Password: 
[root@RHEL4b ~]# updatedb 
[root@RHEL4b ~]# 


Most Linux distributions will schedule the updatedb to run once every day. 


20.3. date 


The date command can display the date, time, time zone and more. 


paul@rhel55 ~S date 
Sat Apr 17 12:44:30 CEST 2010 


A date string can be customised to display the format of your choice. Check the man page 
for more options. 


paul@rhel55 ~$ date +'SA $d-$m-$Y' 
Saturday 17-04-2010 


Time on any Unix is calculated in number of seconds since 1969 (the first second being the 
first second of the first of January 1970). Use date +%s to display Unix time in seconds. 


paul@rhel55 ~$ date +%s 
1271501080 


When will this seconds counter reach two thousand million ? 


paul@rhel55 -$ date -d '1970-01-01 + 2000000000 seconds' 
Wed May 18 04:33:20 CEST 2033 
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20.4. cal 


The cal command displays the current month, with the current day highlighted. 


paul@rhel55 ~S cal 


April 2010 
Su Mo Tu We Th Fr Sa 
2) 9 


AVS C T7 rs MESS Ao 
LL 12 13 14 15 16 17 
te ILS) 20 Bil Ae 23 20 
25 26 27122872930 


You can select any month in the past or the future. 


paul@rhel55 ~$ cal 2 1970 
February 1970 
Su Mo Tu We Th Fr Sa 
T2 38) aM S e 7 
SO MATIA Sen 
Gy a alg aks) aks) 20) 2d 
22 23 2425 26 2722 


20.5. sleep 


The sleep command is sometimes used in scripts to wait a number of seconds. This example 
shows a five second sleep. 


paul@rhel55 ~$ sleep 5 
paul@rhel55 ~$ 
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20.6. time 


The time command can display how long it takes to execute a command. The date command 
takes only a little time. 


paul@rhel55 ~$ time date 
Sat Apr 17 89:08:92] CEST 200 


real 0m0.014s 
user 0m0.008s 
sys Om0.006s 


The sleep 5 command takes five real seconds to execute, but consumes little cpu time. 


paul@rhel55 ~$ time sleep 5 


real 0m5.018s 
user 0m0.005s 
sys Om0.011s 


This bzip2 command compresses a file and uses a lot of cpu time. 


paul@rhel55 ~$ time bzip2 text.txt 


real Om2.368s 
user Om0.847s 
sys Om0.539s 
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20.7. gzip - gunzip 


Users never have enough disk space, so compression comes in handy. The gzip command 
can make files take up less space. 


paul@rhel55 ~$ ls -lh text.txt 

mow-bw-r-—- l1 paul paul 6.4M Apr 17 13:11 text “txt 
paul@rhel55 ~$ gzip text.txt 

paul@rhel55 -$ Is -1h text.txt.gz 

=EW— Ewe I paul paul 760K Apre Sites E 


You can get the original back with gunzip. 


paul@rhel55 ~$ gunzip text.txt.gz 
paul@rhel55 ~$ ls -Ih text.txt 
=rw- rw- r=- J pawl paul 6.4M Apr 17 Sci text. txt 


20.8. zcat - zmore 


Text files that are compressed with gzip can be viewed with zcat and zmore. 


paul@rhel55 -$ head -4 text.txt 

/ 

OPE 

/opt /VBoxGuestAdditions-3.1.6 
/opt/VBoxGuestAdditions-3.1.6/routines.sh 
paul@rhel55 -$ gzip text.txt 

paul@rhel55 ~$ zcat text.txt.gz | head -4 
" 

/opt 

/opt/VBoxGuestAdditions-3.1.6 
/opt/VBoxGuestAdditions-3.1.6/routines.sh 
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20.9. bzip2 - bunzip2 


Files can also be compressed with bzip2 which takes a little more time than gzip, but 
compresses better. 


paul@rhel55 -$ bzip2 text.txt 
paul@rhel55 ~$ ls -lh text.txt.bz2 
SENS ew r= l paul paul 569K Apr 1m 13:11 texti txt- 2. 


Files can be uncompressed again with bunzip2. 


paul@rhel55 ~$ bunzip2 text.txt.bz2 
paul@rhel55 ~$ ls -1h text.txt 
EW GW r- l paul paul 6.4M Apr LS text txt 


20.10. bzcat - bzmore 


And in the same way bzcat and bzmore can display files compressed with bzip2. 


paul@rhel55 ~$ bzip2 text.txt 

paul@rhel55 ~$ bzcat text.txt.bz2 | head -4 
" 

/opt 

/opt/VBoxGuestAdditions-3.1.6 
/opt/VBoxGuestAdditions-3.1.6/routines.sh 
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20.11. practice: basic Unix tools 


1. Explain the difference between these two commands. This question is very important. If 
you don't know the answer, then look back at the shell chapter. 


find /data -name "*.txt" 


find /data -name *.txt 


2. Explain the difference between these two statements. Will they both work when there are 
200 .odf files in /data ? How about when there are 2 million .odf files ? 


find /data name Y= odr e data odr. CE 


find /data/*.odf » data odf.txt 

3. Write a find command that finds all files created after January 30th 2010. 
4. Write a find command that finds all *.odf files created in September 2009. 
5. Count the number of *.conf files in /etc and all its subdirs. 


6. Here are two commands that do the same thing: copy *.odf files to /backup/ . What would 
be a reason to replace the first command with the second ? Again, this is an important 
question. 


cp -r /data/*.odf /backup/ 


find /data -name "*.odf" -exec cp {} /backup/ \; 


7. Create a file called loctest.txt. Can you find this file with locate ? Why not ? How do 
you make locate find this file ? 


8. Use find and -exec to rename all .htm files to .html. 
9. Issue the date command. Now display the date in YYYY/MM/DD format. 


10. Issue the cal command. Display a calendar of 1582 and 1752. Notice anything special ? 
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20.12. solution: basic Unix tools 


1. Explain the difference between these two commands. This question is very important. If 
you don't know the answer, then look back at the shell chapter. 


find /data -name "*.txt" 
find /data -name *.txt 


When *.txt is quoted then the shell will not touch it. The find tool will look in the /data 
for all files ending in .txt. 


When *.txt is not quoted then the shell might expand this (when one or more files that ends 
in .txt exist in the current directory). The find might show a different result, or can result 
in a syntax error. 


2. Explain the difference between these two statements. Will they both work when there are 
200 .odf files in /data ? How about when there are 2 million .odf files ? 
find /data -name "*.odf" > data odf.txt 


find /data/*.odf » data odf.txt 


The first find will output all .odf filenames in /data and all subdirectories. The shell will 
redirect this to a file. 


The second find will output all files named .odf in /data and will also output all files that 
exist in directories named *.odf (in /data). 


With two million files the command line would be expanded beyond the maximum that the 
shell can accept. The last part of the command line would be lost. 
3. Write a find command that finds all files created after January 30th 2010. 


touch -t 201001302359 marker date 
find . -type f -newer marker date 


There is another solution : 
find . -type f -newerat 20100130 23:59:;597 


4. Write a find command that finds all *.odf files created in September 2009. 


touch -t 200908312359 marker start 
touch -t 200910010000 marker end 
find . -type f -name "*.odf" -newer marker start ! -newer marker end 


The exclamation mark ! -newer can be read as not newer. 


5. Count the number of *.conf files in /etc and all its subdirs. 


find /etc -type f -name '*.conf' | wc -1 


6. Here are two commands that do the same thing: copy *.odf files to /backup/ . What would 
be a reason to replace the first command with the second ? Again, this is an important 
question. 


cp -r /data/*.odf /backup/ 
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find /data -name "*.odf" -exec cp {} /backup/ \; 
The first might fail when there are too many files to fit on one command line. 


7. Create a file called loctest.txt. Can you find this file with locate ? Why not ? How do 
you make locate find this file ? 
You cannot locate this with locate because it is not yet in the index. 


updatedb 


8. Use find and -exec to rename all .htm files to .html. 


paul@rhel55 ~$ find . -name '*.htm' 

./one.htm 

./two.htm 

paul@rhel55 =S find! = name "htm exec mv {} (PUN 
paul@rhel55 ~$ find . -name '*.htm*' 

./one.html 

./two.html 


9. Issue the date command. Now display the date in YYYY/MM/DD format. 


date +2%Y/%m/%d 


10. Issue the cal command. Display a calendar of 1582 and 1752. Notice anything special ? 


cal 1582 


The calendars are different depending on the country. Check http://linux-training.be/files/ 
studentfiles/dates.txt 
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Regular expressions are a very powerful tool in Linux. They can be used with a variety of 
programs like bash, vi, rename, grep, sed, and more. 


This chapter introduces you to the basics of regular expressions. 
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21.1. regex versions 


There are three different versions of regular expression syntax: 


BRE: Basic Regular Expressions 
ERE: Extended Regular Expressions 
PRCE: Perl Regular Expressions 


Depending on the tool being used, one or more of these syntaxes can be used. 


For example the grep tool has the -E option to force a string to be read as ERE while -G 
forces BRE and -P forces PRCE. 


Note that grep also has -F to force the string to be read literally. 
The sed tool also has options to choose a regex syntax. 


Read the manual of the tools you use! 
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21.2. grep 


21.2.1. print lines matching a pattern 


grep is a popular Linux tool to search for lines that match a certain pattern. Below are some 
examples of the simplest regular expressions. 


This is the contents of the test file. This file contains three lines (or three newline characters). 


paul@rhel65:~S cat names 
Tania 

Laura 

Valentina 


When grepping for a single character, only the lines containing that character are returned. 


paul@rhel65:~S grep u names 
Laura 

paul@rhel65:~S grep e names 
Valentina 

paul@rhel65:~S grep i names 
Tania 

Valentina 


The pattern matching in this example should be very straightforward; if the given character 
Occurs on a line, then grep will return that line. 


21.2.2. concatenating characters 


Two concatenated characters will have to be concatenated in the same way to have a match. 


This example demonstrates that ia will match Tania but not Valentina and in will match 
Valentina but not Tania. 


paul@rhel65:~S grep a names 
Tania 

Laura 

Valentina 

paul@rhel65:~S grep ia names 
Tania 

paul@rhel65:~S grep in names 
Valentina 

paul@rhel65:~$ 
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21.2.3. one or the other 


PRCE and ERE both use the pipe symbol to signify OR. In this example we grep for lines 
containing the letter i or the letter a. 


paul@debian7:~$ cat list 

Tania 

Laura 

paul@debian7:~$ grep -E 'ila' list 
Tania 

Laura 


Note that we use the -E switch of grep to force interpretion of our string as an ERE. 


We need to escape the pipe symbol in a BRE to get the same logical OR. 


paul@debian7:~$ grep -G 'i|a' Tist 
paul@debian7:~$ grep -G 'iXla' list 
Tania 
Laura 


21.2.4. one or more 


The * signifies zero, one or more occurences of the previous and the + signifies one or more 
of the previous. 


paul@debian7:~$ cat list2 
Ts 
Tol 
lool 
loool 

paul@debian7:~$ grep -E 'o*' list2 


lool 
loool 

paul@debian7:~$ grep -E 'o+' list2 
iiot 
lool 
loool 
paul@debian7:~$ 
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21.2.5. match the end of a string 


For the following examples, we will use this file. 


paul@debian7:~S cat names 
Tania 

Laura 

Valentina 

Fleur 

Floor 


The two examples below show how to use the dollar character to match the end of a string. 


paul@debian7:~$ grep a$ names 
Tania 

Laura 

Valentina 

paul@debian7:~$ grep r$ names 
Fleur 

Floor 


21.2.6. match the start of a string 


The caret character (^) will match a string at the start (or the beginning) of a line. 


Given the same file as above, here are two examples. 


paul@debian7:~$ grep “Val names 
Valentina 

paul@debian7:~$ grep ^F names 
Fleur 

Floor 


Both the dollar sign and the little hat are called anchors in a regex. 
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21.2.7. separating words 


Regular expressions use a \b sequence to reference a word separator. Take for example this 
file: 


paulGdebian7:-$ cat text 
The governer is governing. 
The winter is over. 

Can you get over there? 


Simply grepping for over will give too many results. 


paul@debian7:~$ grep over text 
The governer is governing. 

The winter is over. 

Can you get over there? 


Surrounding the searched word with spaces is not a good solution (because other characters 
can be word separators). This screenshot below show how to use Vb to find only the searched 
word: 


paul@debian7:~$ grep '\bover\b' text 
The winter is over. 

Can you get over there? 
paulGdebian7:-$ 


Note that grep also has a -w option to grep for words. 


paulGdebian7:-$ cat text 

The governer is governing. 

The winter is over. 

Can you get over there? 
paul@debian7:~$ grep -w over text 
The winter is over. 

Can you get over there? 
paul8debian7:-$ 
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21.2.8. grep features 


Sometimes it is easier to combine a simple regex with grep options, than it is to write a more 
complex regex. These options where discussed before: 

grep -i 

grep -v 

grep -w 

grep -A5 

grep -B5 

grep -C5 


21.2.9. preventing shell expansion of a regex 


The dollar sign is a special character, both for the regex and also for the shell (remember 
variables and embedded shells). Therefore it is advised to always quote the regex, this 
prevents shell expansion. 

paul@debian7:~$ grep 'r$' names 


Fleur 
Floor 
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21.3. rename 


21.3.1. the rename command 


On Debian Linux the /usr/bin/rename command is a link to /usr/bin/prename installed by 
the perl package. 


paul@pi ~ $ dpkg -S $(readlink -f $(which rename)) 
perl: /usr/bin/prename 


Red Hat derived systems do not install the same rename command, so this section does not 
describe rename on Red Hat (unless you copy the perl script manually). 


There is often confusion on the internet about the rename command because solutions 
that work fine in Debian (and Ubuntu, xubuntu, Mint, ...) cannot be used in Red Hat 
(and CentOS, Fedora, ...). 


21.3.2. perl 


The rename command is actually a perl script that uses perl regular expressions. The 
complete manual for these can be found by typing perldoc perlrequick (after installing 
perldoc). 


root@pi:~# aptitude install perl-doc 
The following NEW packages will be installed: 

perl-doc 
O packages upgraded, 1 newly installed, 0 to remove and 0 not upgraded. 
Need to get 8,170 kB of archives. After unpacking 13.2 MB will be used. 
Get: 1 http://mirrordirector.raspbian.org/raspbian/ wheezy/main perl-do... 
Fetched 8,170 kB in 19s (412 kB/s) 
Selecting previously unselected package perl-doc. 
(Reading database ... 67121 files and directories currently installed.) 
Unpacking perl-doc (from .../perl-doc_5.14.2-21+rpi2_all.deb) 
Adding 'diversion of /usr/bin/perldoc to /usr/bin/perldoc.stub by perl-doc' 
Processing triggers for man-db 
Setting up peri—doe (5-14 52-21 +rp12) 


root@pi:~# perldoc perlrequick 
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21.3.3. well known syntax 


The most common use of the rename is to search for filenames matching a certain string 
and replacing this string with an other string. 


This is often presented as s/string/other string/ as seen in this example: 


paul@pi ~ $ ls 

abc allfiles.TXT bllfiles.TXT Scratch tennis2.TXT 
abc.conf backup cllfiles.TXT temp.TXT tennis.TXT 
paul@pi ~ $ rename 's/TXT/text/' * 

paul@pi ~ $ ls 

abc allfiles.text bllfiles.text Scratch tennis2.text 
abc.conf backup cllfiles.text  temp.text tennis.text 


And here is another example that uses rename with the well know syntax to change the 
extensions of the same files once more: 


paul@pi ~ $ ls 

abc allfiles.text bllfiles.text Scratch tennis2.text 
abc.conf backup cllfiles.text  temp.text tennis.text 
paul@pi ~ $ rename 's/text/txt/' *.text 

paul@pi ~ $ ls 

abc allfiles.txt bllfiles.txt Scratch tennis2.txt 
abc.conf backup cllfiles.txt  temp.txt  tennis.txt 
paulüpi ~ $ 


These two examples appear to work because the strings we used only exist at the end of the 
filename. Remember that file extensions have no meaning in the bash shell. 


The next example shows what can go wrong with this syntax. 


paul@pi ~ $ touch atxt.txt 

paul@pi ~ $ rename 's/txt/problem/' atxt.txt 

paul@pi ~ $ ls 

abc allfiles.txt backup cllfiles.txt temp.txt tennis.txt 
abc.conf aproblem.txt bllfiles.txt Scratch tennis2.txt 

paulüpi ~ $ 


Only the first occurrence of the searched string is replaced. 


213 


regular expressions 


21.3.4. a global replace 


The syntax used in the previous example can be described as s/regex/replacement/. This 
is simple and straightforward, you enter a regex between the first two slashes and a 
replacement string between the last two. 


This example expands this syntax only a little, by adding a modifier. 


paul@pi ~ $ rename -n 's/TXT/txt/g' aTXT.TXT 
aTXT.TXT renamed as atxt.txt 
paul@pi ~ $ 


The syntax we use now can be described as s/regex/replacement/g where s signifies switch 
and g stands for global. 


Note that this example used the -n switch to show what is being done (instead of actually 
renaming the file). 


21.3.5. case insensitive replace 


Another modifier that can be useful is i. this example shows how to replace a case insensitive 
string with another string. 


paul@debian7:~/files$ 1s 

filel.text  file2.TEXT file3.txt 
paul@debian7:~/files$ rename 's/.text/.txt/i' * 
paul8debian7:-/files$ 1s 

feed txt firle2.txt ases txt 
paul@debian7:~/files$ 


21.3.6. renaming extensions 


Command line Linux has no knowledge of MS-DOS like extensions, but many end users 
and graphical application do use them. 


Here is an example on how to use rename to only rename the file extension. It uses the 
dollar sign to mark the ending of the filename. 


paul@pi ~ $ ls *.txt 

allfiles.txt bllfiles.txt cllfiles.txt  really.txt.txt temp.txt  tennis.txt 
paulpui ~ $ rename s/ Est S pp eris: 

paulbep3 = S 1s * TXI 

allfiles.TXT bllfiles.TXT cllfiles.TXT really.txt.TXT 

temp.TXT tennis.TXT 

paul@pi ~ $ 


Note that the dollar sign in the regex means at the end. Without the dollar sign this 
command would fail on the really.txt.txt file. 
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21.4. sed 


21.4.1. stream editor 


The stream editor or short sed uses regex for stream editing. 


In this example sed is used to replace a string. 


echo Sunday | sed 's/Sun/Mon/' 
Monday 


The slashes can be replaced by a couple of other characters, which can be handy in some 
cases to improve readability. 


echo Sunday | sed 's:Sun:Mon:' 
Monday 
echo Sunday | sed 's Sun Mon ' 
Monday 
echo Sunday | sed 's|Sun|Mon|' 
Monday 


21.4.2. interactive editor 


While sed is meant to be used in a stream, it can also be used interactively on a file. 


paul@debian7:~/files$ echo Sunday > today 
paul@debian7:~/files$ cat today 

Sunday 

paul@debian7:~/files$ sed -i 's/Sun/Mon/' today 
paul@debian7:~/files$ cat today 

Monday 
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21.4.3. simple back referencing 
The ampersand character can be used to reference the searched (and found) string. 


In this example the ampersand is used to double the occurence of the found string. 
echo Sunday | sed 's/Sun/&&/' 
SunSunday 


echo Sunday | sed 's/day/&&/' 
Sundayday 


21.4.4. back referencing 


Parentheses (often called round brackets) are used to group sections of the regex so they 
can leter be referenced. 


Consider this simple example: 
paul@debian7:~$ echo Sunday | sed Ver (Sela) ilias 
Sunnyday 


paul@debian7:~$ echo Sunday | sed 's N(Sun)) Mny M1_' 
Sunny Sunday 


21.4.5. a dot for any character 


In a regex a simple dot can signify any character. 


paul@debian7:~$ echo 2014-04-01 | sed 's/....-..-../YYYY-MM-DD/' 
YYYY-MM-DD 
paul@debian7:~$ echo abcd-ef-gh | sed 's/....-..-../YYYY-MM-DD/' 


21.4.6. multiple back referencing 


When more than one pair of parentheses is used, each of them can be referenced separately 
by consecutive numbers. 


paul@debian7:~$ echo 2014-04-01 | sed 's/N(....9-N(..N)2N C.N) /\TF\24\3/! 
2014404401 
paul@debian7:~$ echo 2014-04-01 | sed 's/N(.... 9 -N(..N)2N C.) /N3:N2: M7" 
01:04:2014 


This feature is called grouping. 
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21.4.7. white space 


The Ys can refer to white space such as a space or a tab. 


This example looks for white spaces (V) globally and replaces them with 1 space. 


paul@debian7:~$ echo -e 'today\tis\twarm' 

today is warm 

paul@debian7:~$ echo -e 'today\tis\twarm' | sed 's As _g 
today is warm 


F 


21.4.8. optional occurrence 


A question mark signifies that the previous is optional. 


The example below searches for three consecutive letter o, but the third o is optional. 


paul@debian7:~$ cat list2 
HIE 
lol 
lool 
loool 

paulGdebian7:-$ grep -E 'ooo?' list2 
lool 
loool 

paul@debian7:~$ cat list2 | sed 's/ooo\?/A/' 
HIE 

lol 

lAl 

lAl 
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21.4.9. exactly n times 


You can demand an exact number of times the oprevious has to occur. 


This example wants exactly three o's. 


paulGdebian7:-$ cat list2 
le 


loool 

paul@debian7:~$ grep -E 'o(3)' list2 

loool 

paul@debian7:~$ cat list2 | sed 's/o\{3\}/A/' 
alae 


paul@debian7:~$ 


21.4.10. between n and m times 


And here we demand exactly from minimum 2 to maximum 3 times. 


paulGdebian7:-$ cat list2 
HIE 


loool 

paul@debian7:~$ grep -E 'o(2,3)' list2 
lool 
loool 

paul@debian7:~$ grep 'o\{2,3\}' list2 
lool 
loool 

paul@debian7:~$ cat list2 | sed 's/o\{2,3\}/A/' 
aE 


lAl 
paul@debian7:~$ 
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21.5. bash history 


The bash shell can also interprete some regular expressions. 


This example shows how to manipulate the exclamation mask history feature of the bash 
shell. 


paul@debian7:~$ mkdir hist 

paul@debian7:~$ cd hist/ 
paul@debian7:~/hist$ touch filel file2 file3 
paul@debian7:~/hist$ ls -1 filel 

rw e r l paul paul 0) Apr 15 22:07 Ee 
paul@debian7:~/hist$ !1 
Is -1 filel 
mw. 1 paul paul 0 Apr I5 22707 filel 
paul@debian7:~/hist$ !1:s/1/3 

Us — delis) 
SES r= l paul paul 0Apr 15 22:07 files 
paul@debian7:~/hist$ 


This also works with the history numbers in bash. 


paul@debian7:~/hist$ history 6 
2089 mkdir hist 
2090 ssl mcr 
2091 touch filel file2 file3 
20192. her eb Gest el 
Zoos aule 
2094 history 6 
paul@debian7:~/hist$ !2092 
ts Eae 
=rw-r-- r= 1 paul paul 0 Apr 15 22707 filel 
paul@debian7:~/hist$ !2092:s/1/2 
ls -1 file2 
rw E- r Il paul paul 0O Apr 15 22:07 fiiez 
paul@debian7:~/hist$ 
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Chapter 22. Introduction to vi 


The vi editor is installed on almost every Unix. Linux will very often install vim (vi 
improved) which is similar. Every system administrator should know vi(m), because it is 
an easy tool to solve problems. 


The vi editor is not intuitive, but once you get to know it, vi becomes a very powerful 


application. Most Linux distributions will include the vimtutor which is a 45 minute lesson 
in vi(m). 
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22.1. command mode and insert mode 


The vi editor starts in command mode. In command mode, you can type commands. Some 
commands will bring you into insert mode. In insert mode, you can type text. The escape 
key will return you to command mode. 


Table 22.1. getting to command mode 


| key action 


| Esc set vi(m) in command mode. 


22.2. start typing (a Ail o O) 


The difference between a A i I o and O is the location where you can start typing. a will 
append after the current character and A will append at the end of the line. i will insert before 
the current character and I will insert at the beginning of the line. o will put you in a new 
line after the current line and O will put you in a new line before the current line. 


Table 22.2. switch to insert mode 


command ¡action 


a start typing after the current character 

A start typing at the end of the current line 

1 start typing before the current character 

I start typing at the start of the current line 

o start typing on a new line after the current line 


start typing on a new line before the current line 
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22.3. replace and delete a character (r x X) 


When in command mode (it doesn't hurt to hit the escape key more than once) you can use 
the x key to delete the current character. The big X key (or shift x) will delete the character 
left of the cursor. Also when in command mode, you can use the r key to replace one single 
character. The r key will bring you in insert mode for just one key press, and will return you 
immediately to command mode. 


Table 22.3. replace and delete 


| command action | 
X delete the character below the cursor 
X delete the character before the cursor 
r replace the character below the cursor 
| p paste after the cursor (here the last deleted character) | 
| xp switch two characters | 


22.4. undo and repeat (u .) 


When in command mode, you can undo your mistakes with u. You can do your mistakes 
twice with . (in other words, the . will repeat your last command). 


Table 22.4. undo and repeat 


command ¡action 


u undo the last action 


repeat the last action 


22.5. cut, copy and paste a line (dd yy p P) 


When in command mode, dd will cut the current line. yy will copy the current line. You can 
paste the last copied or cut line after (p) or before (P) the current line. 


Table 22.5. cut, copy and paste a line 


command [action 
dd cut the current line 
| yy (yank yank) copy the current line 
p paste after the current line 
| P paste before the current line 


224 


Introduction to vi 


22.6. cut, copy and paste lines (3dd 2yy) 


When in command mode, before typing dd or yy, you can type a number to repeat the 
command a number of times. Thus, 5dd will cut 5 lines and 4yy will copy (yank) 4 lines. 
That last one will be noted by vi in the bottom left corner as "4 line yanked". 


Table 22.6. cut, copy and paste lines 


| command ¡action 


3dd cut three lines 


4yy copy four lines 


22.7. start and end of a line (0 or ^ and $) 


When in command mode, the 0 and the caret ^ will bring you to the start of the current line, 
whereas the $ will put the cursor at the end of the current line. You can add 0 and $ to the d 
command, dO will delete every character between the current character and the start of the 
line. Likewise d$ will delete everything from the current character till the end of the line. 
Similarly yO and y$ will yank till start and end of the current line. 


Table 22.7. start and end of line 


command ¡action 


0 jump to start of current line 


jump to start of current line 


$ jump to end of current line 
dO delete until start of line 
d$ delete until end of line 


22.8. join two lines (J) and more 


When in command mode, pressing J will append the next line to the current line. With yyp 
you duplicate a line and with ddp you switch two lines. 


Table 22.8. join two lines 


command ¡action 


J join two lines 


| yyp duplicate a line 


| ddp switch two lines 
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22.9. words (w b) 


When in command mode, w will jump to the next word and b will move to the previous 
word. w and b can also be combined with d and y to copy and cut words (dw db yw yb). 


Table 22.9. words 
command |action 
w forward one word 
b back one word 
3w forward three words 
dw delete one word 
yw yank (copy) one word 


5yb yank five words back 


"dw delete seven words 


22.10. save (or not) and exit (:w :q :q! ) 


Pressing the colon : will allow you to give instructions to vi (technically speaking, typing 
the colon will open the ex editor). :w will write (save) the file, :q will quit an unchanged 
file without saving, and :q! will quit vi discarding any changes. :wq will save and quit and 
is the same as typing ZZ in command mode. 


Table 22.10. save and exit vi 


command action 


:w fname save as fname 


IW save (write) 


:q quit 
:wq save and quit 
ZZ, save and quit 
:q! quit (discarding your changes) 
ww! save (and write to non-writable file!) 


The last one is a bit special. With :w! vi will try to chmod the file to get write permission 
(this works when you are the owner) and will chmod it back when the write succeeds. This 
should always work when you are root (and the file system is writable). 


22.11. Searching (/ ?) 


When in command mode typing / will allow you to search in vi for strings (can be a regular 
expression). Typing /foo will do a forward search for the string foo and typing ?bar will do 
a backward search for bar. 


Table 22.11. searching 


command action 


[string forward search for string 


226 


Introduction to vi 


command 


action 


?string 


n 


backward search for string 


go to next occurrence of search string 


/^string 


forward search string at beginning of line 


/string$ 


forward search string at end of line 


/br[aeio]l 
/\<he\> 


search for bral brel bril and brol 


search for the word he (and not for here or the) 


22.12. replace all ( :1,$ s/foo/bar/g ) 


To replace all occurrences of the string foo with bar, first switch to ex mode with : . Then 
tell vi which lines to use, for example 1,$ will do the replace all from the first to the last 
line. You can write 1,5 to only process the first five lines. The s/foo/bar/g will replace all 
occurrences of foo with bar. 


Table 22.12. replace 

| command action | 
| :4,8 s/foo/bar/g replace foo with bar on lines 4 to 8 | 
| :1,$ s/foo/bar/g replace foo with bar on all lines | 


22.13. reading files (:r :r !cmd) 


When in command mode, :r foo will read the file named foo, :r !foo will execute the 
command foo. The result will be put at the current location. Thus :r Hs will put a listing of 
the current directory in your text file. 


Table 22.13. read files and input 


| command action 
| :rfname — (read) file fname and paste contents 
| ‘r!cmd [execute cmd and paste its output 


22.14. text buffers 


There are 36 buffers in vi to store text. You can use them with the ' character. 


Table 22.14. text buffers 


| command action 
| "add delete current line and put text in buffer a 
"glyy copy seven lines into buffer g 
"ap paste from buffer a 


22.15. multiple files 


You can edit multiple files with vi. Here are some tips. 
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Table 22.15. multiple files 


command action 
vi filel file2 file3 start editing three files 
:args lists files and marks active file 
m start editing the next file 
:e toggle with last edited file 
Tew rewind file pointer to first file 


22.16. abbreviations 


With :ab you can put abbreviations in vi. Use :una to undo the abbreviation. 


Table 22.16. abbreviations 


| command action 
| :ab str long string abbreviate str to be ‘long string' 
| :una str un-abbreviate str 


228 


Introduction to vi 


22.17. key mappings 


Similarly to their abbreviations, you can use mappings with :map for command mode and 
:map! for insert mode. 


This example shows how to set the F6 function key to toggle between set number and set 
nonumber. The «bar? separates the two commands, set number! toggles the state and set 
number? reports the current state. 


:map «F6» :set number!<bar>set number?<CR> 


22.18. setting options 


Some options that you can set in vim. 


:set number ( also try :se nu ) 
:set nonumber 

¿syntax on 

:syntax off 

:set all (list all options) 
:set tabstop-8 

:set tx (CR/LF style endings) 
set NOLX 


You can set these options (and much more) in ~/.vimre for vim or in ~/.exre for standard vi. 


paul@barry:~$ cat -/.vimrc 

set number 

set tabstop-8 

set textwidth-78 

map «F6» :set number!<bar>set number?<CR> 
paul@barry:~$ 
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22.19. practice: vi(m) 


1. Start the vimtutor and do some or all of the exercises. You might need to run aptitude 
install vim on xubuntu. 


2. What 3 key sequence in command mode will duplicate the current line. 


3. What 3 key sequence in command mode will switch two lines' place (line five becomes 
line six and line six becomes line five). 


4. What 2 key sequence in command mode will switch a character's place with the next one. 


5. vi can understand macro's. A macro can be recorded with q followed by the name of 
the macro. So qa will record the macro named a. Pressing q again will end the recording. 
You can recall the macro with €? followed by the name of the macro. Try this example: i 1 
'Escape Key' qa yyp 'Ctrl a' q 5@a (Ctrl a will increase the number with one). 


6. Copy /etc/passwd to your ~/passwd. Open the last one in vi and press Ctrl v. Use the arrow 
keys to select a Visual Block, you can copy this with y or delete it with d. Try pasting it. 


7. What does dwwP do when you are at the beginning of a word in a sentence ? 
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22.20. solution: vi(m) 


1. Start the vimtutor and do some or all of the exercises. You might need to run aptitude 
install vim on xubuntu. 


vimtutor 
2. What 3 key sequence in command mode will duplicate the current line. 
yyP 


3. What 3 key sequence in command mode will switch two lines' place (line five becomes 
line six and line six becomes line five). 


ddp 
4. What 2 key sequence in command mode will switch a character's place with the next one. 
xp 


5. vi can understand macro's. A macro can be recorded with q followed by the name of 
the macro. So qa will record the macro named a. Pressing q again will end the recording. 
You can recall the macro with €? followed by the name of the macro. Try this example: i 1 
'Escape Key' qa yyp 'Ctrl a' q 52a (Ctrl a will increase the number with one). 


6. Copy /etc/passwd to your ~/passwd. Open the last one in vi and press Ctrl v. Use the arrow 
keys to select a Visual Block, you can copy this with y or delete it with d. Try pasting it. 


cp /etc/passwd ~ 
vi passwd 
(press Ctrl-V) 


7. What does dwwP do when you are at the beginning of a word in a sentence ? 


dwwP can switch the current word with the next word. 
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Chapter 23. scripting introduction 


Shells like bash and Korn have support for programming constructs that can be saved as 
scripts. These scripts in turn then become more shell commands. Many Linux commands 
are scripts. User profile scripts are run when a user logs on and init scripts are run when 
a daemon is stopped or started. 


This means that system administrators also need basic knowledge of scripting to understand 
how their servers and their applications are started, updated, upgraded, patched, maintained, 
configured and removed, and also to understand how a user environment is built. 


The goal of this chapter is to give you enough information to be able to read and understand 
scripts. Not to become a writer of complex scripts. 
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23.1. prerequisites 


You should have read and understood part III shell expansion and part IV pipes and 
commands before starting this chapter. 


23.2. hello world 


Just like in every programming course, we start with a simple hello world script. The 
following script will output Hello World. 


echo Hello World 


After creating this simple script in vi or with echo, you'll have to chmod +x hello world 
to make it executable. And unless you add the scripts directory to your path, you'll have to 
type the path to the script for the shell to be able to find it. 


[paul@RHEL4a ~]$ echo echo Hello World > hello world 
[paul@RHEL4a ~]$ chmod +x hello world 

[paul@RHEL4a ~]$ ./hello_world 

Hello World 

[paul@RHEL4a -]$ 


23.3. she-bang 


Let's expand our example a little further by putting #!/bin/bash on the first line of the script. 
The #! is called a she-bang (sometimes called sha-bang), where the she-bang is the first 
two characters of the script. 


#!/bin/bash 
echo Hello World 


You can never be sure which shell a user is running. A script that works flawlessly in bash 
might not work in ksh, csh, or dash. To instruct a shell to run your script in a certain shell, 
you can start your script with a she-bang followed by the shell it is supposed to run in. This 
script will run in a bash shell. 


#!/bin/bash 
echo -n hello 
echo A bash subshell “echo -n hello” 


This script will run in a Korn shell (unless /bin/ksh is a hard link to /bin/bash). The /etc/ 
shells file contains a list of shells on your system. 
#!/bin/ksh 


echo -n hello 
echo a Korn subshell “echo n hello” 
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23.4. comment 


Let's expand our example a little further by adding comment lines. 
#!/bin/bash 

# 

# Hello World Script 


# 
echo Hello World 


23.5. variables 


Here is a simple example of a variable inside a script. 


#!/bin/bash 


# 

# simple variable in script 
# 

varl=4 

echo varl = $varl 


Scripts can contain variables, but since scripts are run in their own shell, the variables do 
not survive the end of the script. 


[paul@RHEL4a ~]$ echo $varl 
[paul@RHEL4a ~]$ ./vars 
varl = 4 


[paul@RHEL4a ~]$ echo $varl 


[paul@RHEL4a ~]$ 


23.6. sourcing a script 


Luckily, you can force a script to run in the same shell; this is called sourcing a script. 


[paul@RHEL4a ~]$ source ./vars 
varl = 4 
[paul@RHEL4a ~]$ echo $varl 
4 
[paul@RHEL4a -]$ 


The above is identical to the below. 


[paul@RHEL4a ~]$ . ./vars 
varl = 4 

[paul@RHEL4a ~]$ echo $varl 
4 

[paul@RHEL4a -]$ 
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23.7. troubleshooting a script 


Another way to run a script in a separate shell is by typing bash with the name of the script 
as a parameter. 


paul@debian6~/test$ bash runme 
42 


Expanding this to bash -x allows you to see the commands that the shell is executing (after 
shell expansion). 


paul@debian6~/test$ bash -x runme 
+ var4=42 

+ echo 42 

42 

paul@debian6~/test$ cat runme 

# the runme script 

var4=42 

echo $var4 

paul@debian6~/test$ 


Notice the absence of the commented (#) line, and the replacement of the variable before 
execution of echo. 


23.8. prevent setuid root spoofing 


Some user may try to perform setuid based script root spoofing. This is a rare but possible 
attack. To improve script security and to avoid interpreter spoofing, you need to add -- after 
the #!/bin/bash, which disables further option processing so the shell will not accept any 
options. 

#!/bin/bash - 


or 
#!/bin/bash -- 


Any arguments after the -- are treated as filenames and arguments. An argument of - is 
equivalent to --. 
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23.9. practice: introduction to scripting 


0. Give each script a different name, keep them for later! 

1. Write a script that outputs the name of a city. 

2. Make sure the script runs in the bash shell. 

3. Make sure the script runs in the Korn shell. 

4. Create a script that defines two variables, and outputs their value. 


5. The previous script does not influence your current shell (the variables do not exist outside 
of the script). Now run the script so that it influences your current shell. 


6. Is there a shorter way to source the script ? 


7. Comment your scripts so that you know what they are doing. 
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23.10. solution: introduction to scripting 


0. Give each script a different name, keep them for later! 


1. Write a script that outputs the name of a city. 
$ echo 'echo Antwerp' » first.bash 
$ chmod +x first.bash 


$ ./first.bash 
Antwerp 


2. Make sure the script runs in the bash shell. 
$ cat first.bash 


#!/bin/bash 
echo Antwerp 


3. Make sure the script runs in the Korn shell. 
$ cat first.bash 


#!/bin/ksh 
echo Antwerp 


Note that while first.bash will technically work as a Korn shell script, the name ending 
in .bash is confusing. 


4. Create a script that defines two variables, and outputs their value. 


$ cat second.bash 
#!/bin/bash 


var33=300 
var42=400 


echo $var33 Svar42 


5. The previous script does not influence your current shell (the variables do not exist outside 
of the script). Now run the script so that it influences your current shell. 


source second.bash 


6. Is there a shorter way to source the script ? 


./second.bash 


7. Comment your scripts so that you know what they are doing. 


$ cat second.bash 
#!/bin/bash 
# script to test variables and sourcing 


# define two variables 
var33=300 
var42=400 


# output the value of these variables 
echo $var33 $var42 
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scripting loops 


24.1. test [ ] 


The test command can test whether something is true or false. Let's start by testing whether 
10 is greater than 55. 


[paul@RHEL4b ~]$ test 10 -gt 55 ; echo $? 
i 
[paul@RHEL4b ~]$ 


The test command returns | if the test fails. And as you see in the next screenshot, test returns 
0 when a test succeeds. 


[paul@RHEL4b -]$ test 56 -gt 55 ; echo $? 


0 
[paul@RHEL4b ~]$ 


If you prefer true and false, then write the test like this. 


[paul@RHEL4b -]$ test 56 -gt 55 && echo true | | echo false 
true 

[paul@RHEL4b -]$ test 6 -gt 55 && echo true | | echo false 
false 


The test command can also be written as square brackets, the screenshot below is identical 
to the one above. 


[paul@RHEL4b -]1$ [ 56 -gt 55 ] && echo true | | echo false 
true 
[paul@RHEL4b -]$ [ 6 -gt 55 ] && echo true | | echo false 
false 


Below are some example tests. Take a look at man test to see more options for tests. 


=d foo |] Does the directory foo exist ? 

-e bar ] Does the file bar exist ? 

'/etc' = SPWD ] Is the string /etc equal to the variable $PWD ? 
$1 != 'secret' ] Is the first parameter different from secret ? 
55 lE Sbarc | Is 55 less than the value of $bar ? 

$foo -ge 1000 ] Is the value of $foo greater or equal to 1000 ? 
tabet < Şbar | Does abc sort before the value of $bar ? 
ESO Is foo a regular file ? 

e pban] Is bar a readable file ? 

foo -nt bar ] Is file foo newer than file bar ? 

-o nounset ] Is the shell option nounset set ? 


Tests can be combined with logical AND and OR. 


paul@RHEL4b:~$ [ 66 -gt 55 -a 66 -lt 500 ] && echo true || echo false 
true 
paul@RHEL4b:~$ [ 66 -gt 55 -a 660 -lt 500 ] && echo true || echo false 
false 
paul@RHEL4b:~$ [ 66 -gt 55 -o 660 -lt 500 ] && echo true || echo false 
true 


241 


scripting loops 


24.2. if then else 


The if then else construction is about choice. If a certain condition is met, then execute 
something, else execute something else. The example below tests whether a file exists, and 
if the file exists then a proper message is echoed. 


#!/bin/bash 


Baz || ae SUS EXE | 

then echo isit.txt exists! 
else echo isit.txt not found! 
fi 


If we name the above script 'choice', then it executes like this. 


[paul@RHEL4a scripts]$ ./choice 

asaite. EXE DOE found! 

[paul@RHEL4a scripts]$ touch isit. txt 
[paul@RHEL4a scripts]$ ./choice 
isit.txt exists! 

[paul@RHEL4a scripts]$ 


24.3. if then elif 


You can nest a new if inside an else with elif. This is a simple example. 


#!/bin/bash 


count=42 
if [ Scount -eq 42 ] 
then 


echo "42 is correct." 
elif [ Scount -gt 42 ] 


then 

echo "Too much." 
else 

echo "Not enough." 
fi 


24.4. for loop 


The example below shows the syntax of a classical for loop in bash. 


forn ai im L2 4 
do 

echo $i 
done 


An example of a for loop combined with an embedded shell. 


#!/bin/ksh 

for counter in `seq 1 20` 

do 
echo counting from 1 to 20, now at Scounter 
Sleep 1 

done 


The same example as above can be written without the embedded shell using the bash 
(from..to) shorthand. 
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#!/bin/bash 
for counter in {1...20} 


do 
echo counting from 1 to 20, now at Scounter 
sleep 1 

done 


This for loop uses file globbing (from the shell expansion). Putting the instruction on the 
command line has identical functionality. 


kahlan@solexp11$ 1s 

count.ksh go.ksh 

kahlantsolexp11$ for file in *.ksh ; do cp $file $file.backup ; done 
kahlantsolexp11$ ls 

count.ksh  count.ksh.backup go.ksh go.ksh.backup 


24.5. while loop 


Below a simple example of a while loop. 


i=100; 

while [ $i -ge 0 ] ; 

do 
echo Counting down, from 100 to 0, now at $i; 
let —— 

done 


Endless loops can be made with while true or while : , where the colon is the equivalent 
of no operation in the Korn and bash shells. 


#!/bin/ksh 
# endless loop 
while 
do 
echo hello 
sleep 1 
done 


24.6. until loop 


Below a simple example of an until loop. 


let i=100; 

uneri ESOS 

do 
echo Counting down, from 100 to 1, now at $i; 
let ==" 

done 
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24.7. practice: scripting tests and loops 


1. Write a script that uses a for loop to count from 3 to 7. 

2. Write a script that uses a for loop to count from 1 to 17000. 

3. Write a script that uses a while loop to count from 3 to 7. 

4. Write a script that uses an until loop to count down from 8 to 4. 

5. Write a script that counts the number of files ending in .txt in the current directory. 


6. Wrap an if statement around the script so it is also correct when there are zero files ending 
in .txt. 


244 


scripting loops 


24.8. solution: scripting tests and loops 


1. Write a script that uses a for loop to count from 3 to 7. 
#!/bin/bash 

for i in 3 d 5617 

do 


echo Counting from 3 to 7, now at $i 
done 


2. Write a script that uses a for loop to count from 1 to 17000. 
#!/bin/bash 

for i in "seq 1 17000” 

do 


echo Counting from 1 to 17000, now at $i 
done 


3. Write a script that uses a while loop to count from 3 to 7. 


#!/bin/bash 


i=3 

While] 

do 
echo Counting from 3 to 7, now at Sa 
let i=i+l 

done 


4. Write a script that uses an until loop to count down from 8 to 4. 


#!/bin/bash 


i-8 

unei p si lt A] 

do 

echo Counting down from 8 to 4, now at $i 
let i=i-1 

done 


5. Write a script that counts the number of files ending in .txt in the current directory. 
#!/bin/bash 


let i=0 
for trke imn r EXE 
do 
Fet itt 
done 
echo "There are $i files ending in .txt" 


6. Wrap an if statement around the script so it is also correct when there are zero files ending 
in .txt. 


#!/bin/bash 
ls artt > /dev/nu ll 2> si 


if [ $? -ne 0 ] 
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then echo "There are 0 files ending in .txt" 
else 

let i=0 

for file in = ext 

do 

let i++ 

done 

echo Yirhere are Sie files ending 1m sts! 

Fa 
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25.1. script parameters 


A bash shell script can have parameters. The numbering you see in the script below 
continues if you have more parameters. You also have special parameters containing the 
number of parameters, a string of all of them, and also the process id, and the last return 
code. The man page of bash has a full list. 


#!/bin/bash 

echo The first argument is $1 
echo The second argument is $2 
echo The third argument is $3 


echo \$ $$ PID of the script 
echo \# $# count arguments 

echo \? $? last return code 
echo \* $* all the arguments 


Below is the output of the script above in action. 


[paul@RHEL4a scripts]$ ./pars one two three 
The first argument is one 

The second argument is two 

The third argument is three 

S 5610) PID of the iseripe 

# 3 count arguments 

? 0 last return code 

* one two three all the arguments 


Once more the same script, but with only two parameters. 


[paul@RHEL4a scripts]$ ./pars 1 2 
The first argument is 1 

The second argument is 2 

The third argument is 

S 5612 PID Of the Script 

# 2 count arguments 

? 0 last return code 

* 1 2 all the arguments 
[paul@RHEL4a scripts]$ 


Here is another example, where we use $0. The $0 parameter contains the name of the script. 


paul@debian6~$ cat myname 

echo this script is called $0 
paul@debian6~$ ./myname 

this script is called ./myname 
paul@debian6~$ mv myname test42 
paul@debian6~$ ./test42 

this script is called ./test42 
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25.2. shift through parameters 


The shift statement can parse all parameters one by one. This is a sample script. 


kahlan@solexp11$ cat shift.ksh 
#!/bin/ksh 


TIE [ Sl == O ] 

then 
echo You have to give at least one parameter. 
exit 1 

fi 


while (( S# )) 
do 
echo You gave me $1 
shift 
done 


Below is some sample output of the script above. 


kahlan@solexp11$ ./shift.ksh one 

You gave me one 

kahlan@solexp11$ ./shift.ksh one two three 1201 "33 42" 
You gave me one 

You gave me two 

You gave me three 

You gave me 1201 

You gave me 33 42 

kahlan@solexp11$ ./shift.ksh 

You have to give at least one parameter. 


25.3. runtime input 


You can ask the user for input with the read command in a script. 


#!/bin/bash 
echo -n Enter a number: 
read number 
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25.4. sourcing a config file 


The source (as seen in the shell chapters) can be used to source a configuration file. 


Below a sample configuration file for an application. 


[paul@RHEL4a scripts]$ cat myApp.conf 
# The config file of myApp 


# Enter the path here 
myAppPath-/var/myApp 


# Enter the number of quines here 
quines-5 


And here an application that uses this file. 


[paul@RHEL4a scripts]$ cat myApp.bash 
#!/bin/bash 

# 

# Welcome to the myApp application 

# 


./myApp. conf 


echo There are $quines quines 


The running application can use the values inside the sourced configuration file. 


[paul@RHEL4a scripts]$ ./myApp.bash 
There are 5 quines 
[paul@RHEL4a scripts]$ 
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25.5. get script options with getopts 


The getopts function allows you to parse options given to a command. The following script 
allows for any combination of the options a, f and z. 


kahlan@solexp11$ cat options.ksh 
#!/bin/ksh 


while getopts ":afz" option; 
do 
case Soption in 
a) 
echo received -a 
tr 
£) 
echo received -f 
ri 
z) 
echo received -z 
ri 
*) 
echo "invalid option -SOPTARG" 
ri 
esac 
done 


This is sample output from the script above. First we use correct options, then we enter twice 
an invalid option. 


kahlan@solexp11$ ./options.ksh 
kahlan@solexp11$ ./options.ksh -af 
received -a 

received -f 

kahlan@solexp11$ ./options.ksh -zfg 
received -z 

received -f 

invalid option g 

kahlan@solexp11$ ./options.ksh a -b -z 
received -a 

invalid option —b 

received -z 
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You can also check for options that need an argument, as this example shows. 


kahlantsolexp11$ cat argoptions.ksh 
#!/bin/ksh 


while getopts ":af:z" option; 
do 
case Soption in 
a) 
echo received -a 


tF 


echo received -f with SOPTARG 
echo received -z 
echo "option -$OPTARG needs an argument" 


echo "invalid option -$OPTARG" 


This is sample output from the script above. 


kahlan@solexp11$ ./argoptions.ksh -a -f hello -z 
received -a 

received -f with hello 

received -z 

kahlan@solexp11$ ./argoptions.ksh -zaf 42 
received -z 

received -a 

received -f with 42 

kahlan@solexp11$ ./argoptions.ksh -zf 
received -z 

option -f needs an argument 


25.6. get shell options with shopt 


You can toggle the values of variables controlling optional shell behaviour with the shopt 
built-in shell command. The example below first verifies whether the cdspell option is set; 
it is not. The next shopt command sets the value, and the third shopt command verifies that 
the option really is set. You can now use minor spelling mistakes in the cd command. The 
man page of bash has a complete list of options. 


paul@laika:~$ shopt -q cdspell ; echo $? 
al 

paul@laika:~$ shopt -s cdspell 
paul@laika:~$ shopt -q cdspell ; echo $? 
0 

paul@laika:~$ cd /Etc 

/etc 
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25.7. practice: parameters and options 


1. Write a script that receives four parameters, and outputs them in reverse order. 


2. Write a script that receives two parameters (two filenames) and outputs whether those 
files exist. 


3. Write a script that asks for a filename. Verify existence of the file, then verify that you 
own the file, and whether it is writable. If not, then make it writable. 


4. Make a configuration file for the previous script. Put a logging switch in the config file, 
logging means writing detailed output of everything the script does to a log file in /tmp. 
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25.8. solution: parameters and options 


1. Write a script that receives four parameters, and outputs them in reverse order. 


echo $4 $3 $2 $1 


2. Write a script that receives two parameters (two filenames) and outputs whether those 
files exist. 


#!/bin/bash 


aU 
then 
else 
fi 


tE II 
then 
else 
fi 


SE SNL | 
echo $1 
echo $1 


SE S 
echo $2 
echo $2 


exists! 
not found! 


exists! 
not found! 


3. Write a script that asks for a filename. Verify existence of the file, then verify that you 
own the file, and whether it is writable. If not, then make it writable. 


4. Make a configuration file for the previous script. Put a logging switch in the config file, 
logging means writing detailed output of everything the script does to a log file in /tmp. 
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more scripting 


26.1. eval 


eval reads arguments as input to the shell (the resulting commands are executed). This allows 
using the value of a variable as a variable. 


paul8deb503:-/test42$ answer-42 
paul8deb503:-/test42$ word-answer 
paul@deb503:~/test42$ eval x=\$$word ; echo $x 
42 


Both in bash and Korn the arguments can be quoted. 


kahlan@solexp11$ answer=42 

kahlantsolexp11$ word=answer 
kahlan@solexp11$ eval "y=\SSword" ; echo Sy 
42 


Sometimes the eval is needed to have correct parsing of arguments. Consider this example 
where the date command receives one parameter 1 week ago. 


paul@debian6~$S dat date="1 week ago" 
Thu Mar 8 2136729 CET 2012 


When we set this command in a variable, then executing that variable fails unless we use 
eval. 


paul@debian6~$ lastweek='dat date="1 week ago"' 
paul@debian6~$ $lastweek 

date: extra operand `ago"' 

Try `date --help' for more information. 
paul@debian6~$ eval $lastweek 

mhu Mar 8 21:36:39 CET 2012 


26.2. (( )) 


The (( )) allows for evaluation of numerical expressions. 


paul@deb503:~/test42$ (( 42 > 33 )) && echo true | | echo false 
true 

paul@deb503:~/test42$ (( 42 > 1201 )) && echo true | | echo false 
false 

paul@deb503:~/test42$ var42=42 

paul@deb503:~/test42$ (( 42 == var42 )) && echo true | | echo false 
true 

paul@deb503:~/test42S (( 42 == $var42 )) && echo true I echo false 
true 

paul@deb503:~/test42$ var42=33 

paul@deb503:~/test42$ (( 42 == var42 )) && echo true | | echo false 
false 
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26.3. let 


The let built-in shell function instructs the shell to perform an evaluation of arithmetic 
expressions. It will return 0 unless the last arithmetic expression evaluates to 0. 


paul@RHEL4b ~]$ let x-"3 + 4" ; echo Sx 


[ 

a 

[paul@RHEL4b ~]$ let x="10 + 100/10" ; echo $x 
20 

[paul@RHEL4b ~]$ let x="10-2+100/10" ; echo $x 
18 

[paul@RHEL4b ~]$ let x="10*2+100/10" ; echo $x 
30 


The shell can also convert between different bases. 


paul@RHEL4b ~]$ let x-"OxFF" ; echo $x 
255 

paul@RHEL4b ~]$ let x="0xC0" ; echo $x 
3L 917 

paul@RHEL4b ~]$ let x="0xA8" ; echo $x 
168 

paul@RHEL4b ~]$ let x="8#70" ; echo Sx 
56 

paul@RHEL4b ~]$ let x="8#77" ; echo $x 
63 

paul@RHEL4b -]$ let x="16#c0O" ; echo Sx 
T92 


There is a difference between assigning a variable directly, or using let to evaluate the 
arithmetic expressions (even if it is just assigning a value). 


kahlan@solexp11$ dec=15 ; oct=017 ; hex=0x0f 
kahlan@solexp11$ echo $dec $oct $hex 

SO OO 

kahlan@solexp11$ let dec=15 ; let oct=017 ; let hex=0x0f 
kahlan@solexp11$ echo $dec $oct Shex 

iS r5 15 
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26.4. case 


You can sometimes simplify nested if statements with a case construct. 


[paul@RHEL4b ~]S . 


What 


/help 


animal did you see ? lion 


You better start running fast! 
[paul@RHEL4b -]$ . 


What 
Don't 


[paul 


#!/bin/bash 


# 


/help 


animal did you see ? dog 
worry, give 
LGRHEL4b -]$ cat help 


it a cookie. 


# Wild Animals Helpdesk Advice 


# 


echo -n "What animal did you see ? " 
read animal 
case Sanimal in 


esac 


IW | "tiger") 
echo "You better start running fast!" 
‘7 
Noam 
echo "Let that mouse go..." 
Pr 
"dog") 
echo "Don't worry, give it a cookie." 
PI 
"chicken" | "goose" | "duck" ) 
echo "Eggs for breakfast!" 
Pr 
"liger") 
echo "Approach and say 'Ah you big fluffy kitty... 
‘7 
"babelfish") 


ec 
Pr 
20 

ec 


ar 


[paul@RHEL4b -]$ 


ho "Did it fall out your ear ?" 


ho "You discovered an unknown animal, name it!" 


258 


more scripting 


26.5. shell functions 


Shell functions can be used to group commands in a logical way. 


kahlan@solexp11$ cat funcs.ksh 
#!/bin/ksh 


function greetings { 

echo Hello World! 

echo and hello to SUSER to! 
} 


echo We will now call a function 
greetings 
echo The end 


This is sample output from this script with a function. 


kahlan@solexp11$ ./funcs.ksh 
We will now call a function 
Hello World! 

and hello to kahlan to! 

The end 


A shell function can also receive parameters. 


kahlantsolexp11$ cat addfunc.ksh 
#!/bin/ksh 


function plus { 

let result="$1 + $2" 
zeho Sil 42 K2 = rosule 
} 


pibus 
plus 20 13 
plus 2022 


This script produces the following output. 


kahlan@solexp11$ ./addfunc.ksh 
SOS 

210) ae ES 33 

AO 42 
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26.6. practice : more scripting 


1. Write a script that asks for two numbers, and outputs the sum and product (as shown here). 


Enter a number: 5 
Enter another number: 2 


Sum: SF AS 
Product: Se Lo 


2. Improve the previous script to test that the numbers are between 1 and 100, exit with an 
error if necessary. 


3. Improve the previous script to congratulate the user if the sum equals the product. 


4. Write a script with a case insensitive case statement, using the shopt nocasematch option. 
The nocasematch option is reset to the value it had before the scripts started. 


5. If time permits (or if you are waiting for other students to finish this practice), take a look 
at Linux system scripts in /etc/init.d and /etc/rc.d and try to understand them. Where does 
execution of a script start in /etc/init.d/samba ? There are also some hidden scripts in ~, we 
will discuss them later. 
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26.7. solution : more scripting 


1. Write a script that asks for two numbers, and outputs the sum and product (as shown here). 


Enter a number: 5 
Enter another number: 2 


Sum: Bae 
Product: 5x 2 = T 
#!/bin/bash 


echo -n "Enter a number : " 
read n1 


echo -n "Enter another number : " 
read n2 


let sum="$n1+$n2" 
let pro="$n1*$n2" 


echo -e "Sum\t: $n1 + $n2 = $sum" 
echo -e “Product\t: sni + Sine = spro" 


2. Improve the previous script to test that the numbers are between 1 and 100, exit with an 


error if necessary. 


echo -n "Enter a number between 1 and 100 : " 
read n1 


Sz |, Seu Sikes db =e Seul eR. 450/00. | 
then 


echo Wrong number... 
exit 1 
fi 


3. Improve the previous script to congratulate the user if the sum equals the product. 


if [ $sum -eq Spro ] 
then echo Congratulations $sum == $pro 
a 


4. Write a script with a case insensitive case statement, using the shopt nocasematch option. 
The nocasematch option is reset to the value it had before the scripts started. 


#!/bin/bash 

# 

# Wild Animals Case Insensitive Helpdesk Advice 
# 


if shopt -q nocasematch; then 
nocase=yes; 

else 
nocase=no; 
shopt -s nocasematch; 

feu 


echo -n "What animal did you see ? " 
read animal 


case $animal in 
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"lion" | "tiger") 
echo "You better start running fast!" 


echo "Let that mouse go..." 


“dog D 
echo "Don't worry, give it a cookie." 


"chicken" | "goose" | "duck" ) 
echo "Eggs for breakfast!" 
Pr 

"liger") 

echo "Approach and say 'Ah you big fluffy kitty.'" 
‘7 
"babelfish") 

echo "Did it fall out your ear ?" 


echo "You discovered an unknown animal, name it!" 


if [ nocase = yes ] ; then 
shopt -s nocasematch; 
else 
shopt -u nocasematch; 
fi 


5. If time permits (or if you are waiting for other students to finish this practice), take a look 
at Linux system scripts in /etc/init.d and /etc/rc.d and try to understand them. Where does 
execution of a script start in /etc/init.d/samba ? There are also some hidden scripts in ~, we 
will discuss them later. 
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Chapter 27. introduction to users 


This little chapter will teach you how to identify your user account on a Unix computer using 
commands like who am i, id, and more. 


In a second part you will learn how to become another user with the su command. 


And you will learn how to run a program as another user with sudo. 
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27.1. whoami 


The whoami command tells you your username. 
[paul@centos7 ~]$ whoami 


paul 
[paul@centos7 ~]$ 


27.2. who 


The who command will give you information about who is logged on the system. 


[paul@centos7 ~]$ who 


root pts/0 2014=10=10 23707 (O! 10433 TOT) 
paul pts/1 OLA qn (IHR SEM) 
laura pts/2 20T14—-T0—-10 23:34 (10.104.33.96) 
tania pts/3 2014=10=10 29:39 (10. 104.33791) 


[paul@centos7 ~]$ 


27.3. who ami 


With who ami the who command will display only the line pointing to your current session. 
[paul@centos7 -]$ who am i 


paul pts/1 2014=10=10 23:730 (10. 104.33. Ob 
[paul@centos7 ~]$ 


27.4. W 


The w command shows you who is logged on and what they are doing. 


[paul@centos7 ~] $ w 
23:34:07 up 31 min, 2 users, load average: 0.00, 0.01, 0.02 
USER ERY LOGING IDLE dOPU PCPU WHAT 
root pts/0 23:07 1:5500s 30/04 s: OO SUE OD 
paul pts/1 2.59 SO) 7.00s 0.00s 0.00s w 


[paul@centos7 ~]$ 


27.5. id 


The id command will give you your user id, primary group id, and a list of the groups that 
you belong to. 


paul@debian7:~$ id 
uid=1000 (paul) gid=1000 (paul) groups=1000 (paul) 


On RHEL/CentOS you will also get SELinux context information with this command. 
[root@centos7 ~]# id 


uid=0(root) gid=0(root) groups=0(root) context=unconfined_u:unconfined_r\ 
:unconfined t:s0-s0:c0.c1023 
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27.6. su to another user 


The su command allows a user to run a shell as another user. 


laura@debian7:~S su tania 
Password: 
tanialdebian7:/home/laura$ 


27.7. su to root 


Yes you can also su to become root, when you know the root password. 


laura@debian7:~$ su root 
Password: 
root@debian7:/home/laura# 


27.8. su as root 


You need to know the password of the user you want to substitute to, unless your are logged 
in as root. The root user can become any existing user without knowing that user's password. 


root@debian7:~# id 

uid=0 (root) gid=0(root) groups=0 (root) 
root@debian7:~# su — valentina 
valentina@debian7:~$ 


27.9. su - $username 


By default, the su command maintains the same shell environment. To become another user 


and also get the target user's environment, issue the su - command followed by the target 
username. 


root@debian7:~# su laura 
laura@debian7:/rootS exit 
exit 

root@debian7:~# su — laura 
laura@debian7:~$ pwd 
/home/laura 


27.10. su - 


When no username is provided to su or su -, the command will assume root is the target. 
tania@debian7:~$ su - 


Password: 
root@debian7:~# 
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27.11. run a program as another user 


The sudo program allows a user to start a program with the credentials of another user. 
Before this works, the system administrator has to set up the /etc/sudoers file. This can be 
useful to delegate administrative tasks to another user (without giving the root password). 


The screenshot below shows the usage of sudo. User paul received the right to run useradd 
with the credentials of root. This allows paul to create new users on the system without 
becoming root and without knowing the root password. 


First the command fails for paul. 
paul@debian7:~$ /usr/sbin/useradd -m valentina 


useradd: Permission denied. 
useradd: cannot lock /etc/passwd; try again later. 


But with sudo it works. 
paul@debian7:~$ sudo /usr/sbin/useradd -m valentina 


[sudo] password for paul: 
paul@debian7:~$ 


27.12. visudo 


Check the man page of visudo before playing with the /etc/sudoers file. Editing the sudoers 
is out of scope for this fundamentals book. 
paul@rhel65:~S apropos visudo 


visudo (8) - edit the sudoers file 
paul@rhel65:~$ 
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27.13. sudo su - 


On some Linux systems like Ubuntu and Xubuntu, the root user does not have a password 
set. This means that it is not possible to login as root (extra security). To perform tasks as 
root, the first user is given all sudo rights via the /etc/sudoers. In fact all users that are 
members of the admin group can use sudo to run all commands as root. 


root@laika:~# grep admin /etc/sudoers 
# Members of the admin group may gain root privileges 
gadmin ALL- (ALL) ALL 


The end result of this is that the user can type sudo su - and become root without having to 
enter the root password. The sudo command does require you to enter your own password. 
Thus the password prompt in the screenshot below is for sudo, not for su. 


paul@laika:~$ sudo su - 
Password: 
root@laika:~# 


27.14. sudo logging 


Using sudo without authorization will result in a severe warning: 
paul@rhel65:~S sudo su - 


We trust you have received the usual lecture from the local System 
Administrator. It usually boils down to these three things: 


#1) Respect the privacy of others. 
#2) Think before you type. 
#3) With great power comes great responsibility. 


[sudo] password for paul: 
paul is not in the sudoers file. This incident will be reported. 
paul@rhel65:~$ 


The root user can see this in the /var/log/secure on Red Hat and in /var/log/auth.log on 
Debian). 


root@rhel65:~# tail /var/log/secure | grep sudo | tr -s ' ' 

Apr 13 16:03:42 rhel65 sudo: paul : user NOT in sudoers ; TTY=pts/0 ; PWD=\ 
/home/paul ; USER-root ; COMMAND-/bin/su - 

root@rhel65:~# 
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27.15. practice: introduction to users 


1. Run a command that displays only your currently logged on user name. 
2. Display a list of all logged on users. 


3. Display a list of all logged on users including the command they are running at this very 
moment. 


4. Display your user name and your unique user identification (userid). 


5. Use su to switch to another user account (unless you are root, you will need the password 
of the other account). And get back to the previous account. 


6. Now use su - to switch to another user and notice the difference. 
Note that su - gets you into the home directory of Tania. 


7. Try to create a new user account (when using your normal user account). this should fail. 
(Details on adding user accounts are explained in the next chapter.) 


8. Now try the same, but with sudo before your command. 
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27.16. solution: introduction to users 


]. Run a command that displays only your currently logged on user name. 


laura@debian7:~S whoami 
laura 

laura@debian7:~$ echo $USER 
laura 


2. Display a list of all logged on users. 


laura@debian7:~$ who 
laura pts/0 ENO AM) SS (pma 22 OS Sao) 
laura@debian7:~$ 


3. Display a list of all logged on users including the command they are running at this very 
moment. 


lauraldebian7:-$ w 
OW 347302) up sor miT, 2 users, load average: 0.00, 0.00, 0.00 


USER TTY FROM LOGINQ IDLE JCPU PCPU WHAT 
root pts/0 OR OAR eS ROT 0/7» 8:810) 6.00s 0.04s 0.00s w 
root pts/1 3500) eo Sto NON 07:46 6.00s 0.01s 0.00s sleep 42 


laura@debian7:~$ 


4. Display your user name and your unique user identification (userid). 


laura@debian7:~$ id 
uid=1005(laura) gid=1007(laura) groups=1007 (laura) 
laura@debian7:~$ 


5. Use su to switch to another user account (unless you are root, you will need the password 
of the other account). And get back to the previous account. 


laura@debian7:~S su tania 

Password: 

tania@debian7:/home/laura$ id 

uid=1006(tania) gid=1008(tania) groups=1008 (tania) 
tania@debian7:/home/laura$ exit 

laura@debian7:~S$ 


6. Now use su - to switch to another user and notice the difference. 


lauraldebian7:-$ su — tania 
Password: 

tanialdebian7:-$ pwd 
/home/tania 
tania@debian7:~$ logout 
laura@debian7:~$ 


Note that su - gets you into the home directory of Tania. 
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7. Try to create a new user account (when using your normal user account). this should fail. 
(Details on adding user accounts are explained in the next chapter.) 


laura@debian7:~$ useradd valentina 

-su: useradd: command not found 

laura@debian7:~$ /usr/sbin/useradd valentina 
useradd: Permission denied. 

useradd: cannot lock /etc/passwd; try again later. 


It is possible that useradd is located in /sbin/useradd on your computer. 


8. Now try the same, but with sudo before your command. 
laura@debian7:~$ sudo /usr/sbin/useradd valentina 
[sudo] password for laura: 


laura is not in the sudoers file. This incident will be reported. 
lauraldebian7:-$ 


Notice that laura has no permission to use the sudo on this system. 
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This chapter will teach you how to use useradd, usermod and userdel to create, modify 
and remove user accounts. 


You will need root access on a Linux computer to complete this chapter. 


274 


user management 


28.1. user management 


User management on Linux can be done in three complementary ways. You can use the 
graphical tools provided by your distribution. These tools have a look and feel that depends 
on the distribution. If you are a novice Linux user on your home system, then use the 
graphical tool that is provided by your distribution. This will make sure that you do not run 
into problems. 


Another option is to use command line tools like useradd, usermod, gpasswd, passwd and 
others. Server administrators are likely to use these tools, since they are familiar and very 
similar across many different distributions. This chapter will focus on these command line 
tools. 


A third and rather extremist way is to edit the local configuration files directly using vi (or 
vipw/vigr). Do not attempt this as a novice on production systems! 


28.2. letc/passwd 


The local user database on Linux (and on most Unixes) is /etc/passwd. 


[root@RHEL5 -]4 tail /etc/passwd 

inge:x:518:524:art dealer:/home/inge:/bin/ksh 
ann:x:519:525:flute player:/home/ann:/bin/bash 
frederik:x:520:526:rubius poet:/home/frederik:/bin/bash 
Steven:x:521:527:roman emperor:/home/steven:/bin/bash 
pascale:x:522:528:artist:/home/pascale:/bin/ksh 
geert:x:524:530:kernel developer:/home/geert:/bin/bash 
wim:x:525:531:master damuti:/home/wim:/bin/bash 
sandra:x:526:532:radish stresser:/home/sandra:/bin/bash 
annelies:x:527:533:sword fighter:/home/annelies:/bin/bash 
laura:x:528:534:art dealer:/home/laura:/bin/ksh 


As you can see, this file contains seven columns separated by a colon. The columns contain 
the username, an x, the user id, the primary group id, a description, the name of the home 
directory, and the login shell. 


More information can be found by typing man 5 passwd. 


[root@RHEL5 ~]# man 5 passwd 


28.3. root 


The root user also called the superuser is the most powerful account on your Linux system. 
This user can do almost anything, including the creation of other users. The root user always 
has userid 0 (regardless of the name of the account). 


[root@RHEL5 ~]# head -1 /etc/passwd 
root x O 0 ao ot TOO: bin) basin 
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28.4. useradd 


You can add users with the useradd command. The example below shows how to add a 
user named yanina (last parameter) and at the same time forcing the creation of the home 
directory (-m), setting the name of the home directory (-d), and setting a description (-c). 


[root@RHEL5 -]4 useradd -m -d /home/yanina -c "yanina wickmayer" yanina 
[root@RHEL5 ~]# tail -1 /etc/passwd 
yanina:x:529:529:yanina wickmayer:/home/yanina:/bin/bash 


The user named yanina received userid 529 and primary group id 529. 


28.5. /etc/default/useradd 


Both Red Hat Enterprise Linux and Debian/Ubuntu have a file called /etc/default/useradd 
that contains some default user options. Besides using cat to display this file, you can also 
use useradd -D. 


[root@RHEL4 ~]# useradd -D 
GROUP=100 

HOME=/home 

INACTIVE--1 

EXPIRE- 

SHELL-/bin/bash 
SKEL-/etc/skel 


28.6. userdel 


You can delete the user yanina with userdel. The -r option of userdel will also remove the 
home directory. 


[root@RHEL5 ~]# userdel -r yanina 


28.7. usermod 


You can modify the properties of a user with the usermod command. This example uses 
usermod to change the description of the user harry. 


[root@RHEL4 ~]# tail -1 /etc/passwd 
harry:x:516:520:harry potter:/home/harry:/bin/bash 
[root@RHEL4 ~]# usermod -c 'wizard' harry 
[root@RHEL4 ~]# tail -1 /etc/passwd 
harry:x:516:520:wizard:/home/harry:/bin/bash 
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28.8. creating home directories 


The easiest way to create a home directory is to supply the -m option with useradd (it is 
likely set as a default option on Linux). 


A less easy way is to create a home directory manually with mkdir which also requires 
setting the owner and the permissions on the directory with chmod and chown (both 
commands are discussed in detail in another chapter). 


[root@RHEL5 ~]# mkdir /home/laura 

[root@RHEL5 ~]# chown laura:laura /home/laura 
[root@RHEL5 ~]# chmod 700 /home/laura 

[root@RHEL5 ~]# ls -ld /home/laura/ 

drwx------ 2 laura laura 4096 Jun 24 15:17 /home/laura/ 


28.9. /etc/skel/ 


When using useradd the -m option, the /etc/skel/ directory is copied to the newly created 
home directory. The /etc/skel/ directory contains some (usually hidden) files that contain 
profile settings and default values for applications. In this way /etc/skel/ serves as a default 
home directory and as a default user profile. 


[root@RHEL5 ~]# ls -la /etc/skel/ 

total 48 

drwxr-xr-x 2 root root 4096 Apr OO Ra. 

drwxr- xr oo root i2283 mz LESS ee 

rw e- r il Poot root 24 Jul 12 2006 .bash logout 

Say ij=o=.== il ieee male 176 Jul 12 2006 .bash_profile 
al 


SEW ee root root 124 Jul 12 2006 -bashre 


28.10. deleting home directories 


The -r option of userdel will make sure that the home directory is deleted together with the 
user account. 


[root@RHEL5 ~]# ls -ld /home/wim/ 

elor- ===- 2 wim wim 4096 Jun 24 15:19 /home/wim/ 
[root@RHEL5 ~] # userdel -r wim 

[root@RHEL5 ~]# ls -ld /home/wim/ 

ls: /home/wim/: No such file or directory 
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28.11. login shell 


The /etc/passwd file specifies the login shell for the user. In the screenshot below you can 
see that user annelies will log in with the /bin/bash shell, and user laura with the /bin/ksh 
shell. 


[root@RHEL5 ~]# tail -2 /etc/passwd 
annelies:x:527:533:sword fighter:/home/annelies:/bin/bash 
laura:x:528:534:art dealer:/home/laura:/bin/ksh 


You can use the usermod command to change the shell for a user. 


[root@RHEL5 ~]# usermod -s /bin/bash laura 
[root@RHEL5 ~]# tail -1 /etc/passwd 
laura:x:528:534:art dealer:/home/laura:/bin/bash 


28.12. chsh 


Users can change their login shell with the chsh command. First, user harry obtains a list of 
available shells (he could also have done a cat /etc/shells) and then changes his login shell 
to the Korn shell (/bin/ksh). At the next login, harry will default into ksh instead of bash. 


[laura@centos7 -]$ chsh -1 
/bin/sh 

/bin/bash 

/ Sbin/nologin 
/usr/bin/sh 
/usr/bin/bash 
/usr/sbin/nologin 
/bin/ksh 

/bin/tcsh 

/bin/csh 
[laura@centos7 ~]$ 


Note that the -l option does not exist on Debian and that the above screenshot assumes that 
ksh and csh shells are installed. 


The screenshot below shows how laura can change her default shell (active on next login). 
[laura@centos7 ~]$ chsh -s /bin/ksh 
Changing shell for laura. 


Password: 
Shell changed. 
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28.13. practice: user management 


1. Create a user account named serena, including a home directory and a description (or 
comment) that reads Serena Williams. Do all this in one single command. 


2. Create a user named venus, including home directory, bash shell, a description that reads 
Venus Williams all in one single command. 


3. Verify that both users have correct entries in /etc/passwd, /etc/shadow and /etc/group. 
4. Verify that their home directory was created. 
5. Create a user named einstime with /bin/date as his default logon shell. 


7. What happens when you log on with the einstime user ? Can you think of a useful real 
world example for changing a user's login shell to an application ? 


8. Create a file named welcome.txt and make sure every new user will see this file in their 
home directory. 


9. Verify this setup by creating (and deleting) a test user account. 


10. Change the default login shell for the serena user to /bin/bash. Verify before and after 
you make this change. 
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28.14. solution: user management 


]. Create a user account named serena, including a home directory and a description (or 
comment) that reads Serena Williams. Do all this in one single command. 


root@debian7:~# useradd -m -c 'Serena Williams' serena 


2. Create a user named venus, including home directory, bash shell, a description that reads 
Venus Williams all in one single command. 


root@debian7:~# useradd -m -c "Venus Williams" -s /bin/bash venus 


3. Verify that both users have correct entries in /etc/passwd, /etc/shadow and /etc/group. 


root@debian7:~# tail -2 /etc/passwd 
serena:x:1008:1010:Serena Williams:/home/serena:/bin/sh 
venus:x:1009:1011:Venus Williams: /home/venus:/bin/bash 
root@debian7:~# tail -2 /etc/shadow 
serenas i6 SO Ios 

venus 116398705 99999: 

root@debian7:~# tail -2 /etc/group 

serena:x:1010: 

venus:x:1011: 


4. Verify that their home directory was created. 
root@debian7:~# ls -lrt /home | tail -2 
drwxr-xr-x 2 serena serena 4096 Oct 15 10:50 serena 


drwxr-xr-x 2 venus venus 4096 Oct 15 10:59 venus 
root@debian7:~# 


5. Create a user named einstime with /bin/date as his default logon shell. 


root@debian7:~# useradd -s /bin/date einstime 


Or even better: 


root@debian7:~# useradd -s $(which date) einstime 


7. What happens when you log on with the einstime user ? Can you think of a useful real 
world example for changing a user's login shell to an application ? 
root@debian7:~# su — einstime 


Wed Oct 15 11:05:56 UTC 2014 # You get the output of the date command 
root@debian7:~# 


It can be useful when users need to access only one application on the server. Just logging 
in opens the application for them, and closing the application automatically logs them out. 
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8. Create a file named welcome.txt and make sure every new user will see this file in their 
home directory. 


root@debian7:~# echo Hello > /etc/skel/welcome.txt 


9. Verify this setup by creating (and deleting) a test user account. 


root@debian7:~# useradd -m test 
root@debian7:~# ls -1 /home/test 
total 4 
EW- EEr 1 test test 6 Oct 15 11:16 welcome.txt 
root@debian7:~# userdel -r test 
root@debian7:~# 


10. Change the default login shell for the serena user to /bin/bash. Verify before and after 
you make this change. 


root@debian7:~# grep serena /etc/passwd 
serena:x:1008:1010:Serena Williams:/home/serena:/bin/sh 
root@debian7:~# usermod -s /bin/bash serena 
root@debian7:~# grep serena /etc/passwd 
serena:x:1008:1010:Serena Williams:/home/serena:/bin/bash 
rootftdebian7:-+ 
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Chapter 29. user passwords 


This chapter will tell you more about passwords for local users. 


Three methods for setting passwords are explained; using the passwd command, using 
openssel passwd, and using the crypt function in a C program. 


The chapter will also discuss password settings and disabling, suspending or locking 
accounts. 
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29.1. passwd 


Passwords of users can be set with the passwd command. Users will have to provide their 
old password before twice entering the new one. 


[tania@centos7 ~]$ passwd 

Changing password for user tania. 

Changing password for tania. 

(current) UNIX password: 

New password: 

BAD PASSWORD: The password is shorter than 8 characters 
New password: 

BAD PASSWORD: The password is a palindrome 

New password: 

BAD PASSWORD: The password is too similar to the old one 
passwd: Have exhausted maximum number of retries for service 


As you can see, the passwd tool will do some basic verification to prevent users from using 
too simple passwords. The root user does not have to follow these rules (there will be 
a warning though). The root user also does not have to provide the old password before 
entering the new password twice. 


root@debian7:~# passwd tania 

Enter new UNIX password: 

Retype new UNIX password: 

passwd: password updated successfully 


29.2. shadow file 


User passwords are encrypted and kept in /etc/shadow. The /etc/shadow file is read only 
and can only be read by root. We will see in the file permissions section how it is possible 
for users to change their password. For now, you will have to know that users can change 
their password with the /usr/bin/passwd command. 


[root@centos7 ~]# tail -4 /etc/shadow 
paul:$6$ikp2Xta5BT.Tml.p$2TZjNnOYNNOKpwLJqoGJbVsZG5/Fti8ovBRd.VzRbiDSl7TEqN 
IaSMH.TeBKnTS/SjlMruW8qffCOJNORW.BTW1:16338:0:99999:7:::: 
tania:$6$8Z2/zovxj$9qvoqI8i9KIrmN.k4EQwAF5ryz5yzNwEvYjAa9L5XVXQu.z4DlpvMREHN 
eOpOzvRngFdKkVj17H5ST.c79HDZw0:16356:0:99999:7::: 
laura:$6$glDuTY5eS$/NYYWLxfHgZFWeoujaXSMCR.Mz.lGOxtcxFocFVJNb98nbTPhWEFXfKWGN 
SyYhlWCv6763Wq54.w24Yr3uAZBOm/:16356:0:99999:7::: 
valentina:$6$jrZa6PVIS$1uQgqR6En9mZB6mKJ3LXRBA4CnFko6LRhbh.v4iqUk9MVreuillv7N 
GxHOUDSKAON55ZRNhGHa6T20uFnVno/001:16356:0:99999:7::: 

[root@centos7 ~]# 


The /etc/shadow file contains nine colon separated columns. The nine fields contain (from 
left to right) the user name, the encrypted password (note that only inge and laura have an 
encrypted password), the day the password was last changed (day 1 is January 1, 1970), 
number of days the password must be left unchanged, password expiry day, warning number 
of days before password expiry, number of days after expiry before disabling the account, 
and the day the account was disabled (again, since 1970). The last field has no meaning yet. 


All the passwords in the screenshot above are hashes of hunter2. 
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29.3. encryption with passwd 


Passwords are stored in an encrypted format. This encryption is done by the crypt function. 
The easiest (and recommended) way to add a user with a password to the system is to add 
the user with the useradd -m user command, and then set the user's password with passwd. 


[root@RHEL4 ~]# useradd -m xavier 

[root@RHEL4 ~]# passwd xavier 

Changing password for user xavier. 

New UNIX password: 

Retype new UNIX password: 

passwd: all authentication tokens updated successfully. 
[root@RHEL4 ~]# 


29.4. encryption with openssl 


Another way to create users with a password is to use the -p option of useradd, but that 
option requires an encrypted password. You can generate this encrypted password with the 
openssl passwd command. 


The openssl passwd command will generate several distinct hashes for the same password, 
for this it uses a salt. 


paul@rhel65:~$ openssl passwd hunter2 
86jcUNlinGDFpY 
paul@rhel65:~$ openssl passwd hunter2 
Yj7mDO9OAnvq6 
paul@rhel65:~$ openssl passwd hunter2 
YqDcJeGoDbzKA 
paul@rhel65:~$ 


This salt can be chosen and is visible as the first two characters of the hash. 


paul@rhel65:~$ openssl passwd -salt 42 hunter2 
42ZrbtP1Ze8G. 

paulGrhel65:-$ openssl passwd -salt 42 hunter2 
42ZrbtP1Ze8G. 

paulGrhel65:-$ openssl passwd -salt 42 hunter2 
42ZrbtP1Ze8G. 

paul@rhel65:~$ 


This example shows how to create a user with password. 


root@rhel65:~# useradd -m -p $(openssl passwd hunter2) mohamed 


Note that this command puts the password in your command history! 
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29.5. encryption with crypt 


A third option is to create your own C program using the crypt function, and compile this 
into a command. 


paul@rhel65:~S cat MyCrypt.c 
#include <stdio.h> 

#define __USE_XOPEN 

#include <unistd.h> 


ine main(int arge, char** argy) 
{ 
if (argc==3) 
{ 
printf (Mesa crypt (argyll, argvi2i)); 
} 


else 
i 

printf ("Usage: MyCrypt $password $salt\n" ); 
} 


return 0; 


} 


This little program can be compiled with gcc like this. 


paul@rhel65:~$ gcc MyCrypt.c -o MyCrypt -lcrypt 


To use it, we need to give two parameters to MyCrypt. The first is the unencrypted password, 
the second is the salt. The salt is used to perturb the encryption algorithm in one of 4096 
different ways. This variation prevents two users with the same password from having the 
same entry in /etc/shadow. 


paul@rhel65:~$ ./MyCrypt hunter2 42 
42ZrbtP1Ze8G. 
paul@rhel65:~$ ./MyCrypt hunter2 33 
33d6taYSiEUXI 


Did you notice that the first two characters of the password are the salt? 


The standard output of the crypt function is using the DES algorithm which is old and can 
be cracked in minutes. A better method is to use md5 passwords which can be recognized 
by a salt starting with $1$. 


paul@rhel65:~$ ./MyCrypt hunter2 '$1$42' 

$1$42$716Y3xT5282XmZrtDOF9f0 

paul@rhel65:~S ./MyCrypt hunter2 '$6$42' 
$6$42$0qgFFAVnI3gTSYGOyI9TZWX9cpyOzwIop7HwpGlLLEsNBiMr4w6OvLX1KDa./UpwXfrFkli... 


The md5 salt can be up to eight characters long. The salt is displayed in /etc/shadow between 
the second and third $, so never use the password as the salt! 


paul@rhel65:~S ./MyCrypt hunter2 '$1$hunter2' 
$1$hunter2S$YVxrxDmidq7Xf8Gdt6qM2. 
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29.6. /etc/login.defs 


The /etc/login.defs file contains some default settings for user passwords like password 
aging and length settings. (You will also find the numerical limits of user ids and group ids 
and whether or not a home directory should be created by default). 


root@rhel65:~# grep “PASS /etc/login.defs 
PASS MAX DAYS 9199919 

PASS MIN DAYS 0 

PASS_MIN_LEN 5 

PASS WARN AGE F 


Debian also has this file. 


root@debian7:~# grep PASS /etc/login.defs 


+ | PASS MAX DAYS Maximum number of days a password may be used. 
# | PASS MIN DAYS Minimum number of days allowed between password changes. 
# PASS WARN AGE Number of days warning given before a password expires. 


PASS MAX DAYS 919191919, 
PASS_MIN_DAYS 0 
PASS WARN AGE 7 
#PASS_CHANGE_TRIES 
#PASS_ALWAYS_WARN 
#PASS_MIN_LEN 
#PASS_MAX_LEN 

# NO_PASSWORD_CONSOLE 
root@debian7:~# 


29.7. chage 


The chage command can be used to set an expiration date for a user account (-E), set a 
minimum (-m) and maximum (-M) password age, a password expiration date, and set the 
number of warning days before the password expiration date. Much of this functionality is 
also available from the passwd command. The -l option of chage will list these settings for 


a user. 


root@rhel65:~# chage -1 paul 

Last password change 

Password expires 

Password inactive 

Account expires 

Minimum number of days between password change 
Maximum number of days between password change 
Number of days of warning before password expires 
root@rhel65:~# 


Mar 27, 2014 
never 

never 

never 

0 

99999 

7 
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29.8. disabling a password 


Passwords in /etc/shadow cannot begin with an exclamation mark. When the second field 
in /etc/passwd starts with an exclamation mark, then the password can not be used. 


Using this feature is often called locking, disabling, or suspending a user account. Besides 
vi (or vipw) you can also accomplish this with usermod. 


The first command in the next screenshot will show the hashed password of laura in /etc/ 
shadow. The next command disables the password of laura, making it impossible for Laura 
to authenticate using this password. 


root@debian7:~# grep laura /etc/shadow | cut -c1-70 
laura:$6$JYj4JZqp$stwwWACp3OtE1R2aZuE87j.nbW.puDkNUYVk7mCHfCVMa3CoDUJV 
root@debian7:~# usermod -L laura 


As you can see below, the password hash is simply preceded with an exclamation mark. 


root@debian7:~# grep laura /etc/shadow | cut -c1-70 
laura: !$6$JY34JZqp$stwwWACp30tE1R2aZuE873.nbW. puDkNUYVk7mCHfCVMa3CoDUJ 
root @debian7:~# 


The root user (and users with sudo rights on su) still will be able to su into the laura account 
(because the password is not needed here). Also note that laura will still be able to login 
if she has set up passwordless ssh! 


root@debian7:~# su — laura 
laura@debian7:~$ 


You can unlock the account again with usermod -U. 


root@debian7:~# usermod -U laura 
root@debian7:~# grep laura /etc/shadow | cut -c1-70 
laura:$6$JYj4JZqp$stwwWACp3OtE1R2aZuE87j.nbW.puDkNUYVk7mCHfCVMa3CoDUJV 


Watch out for tiny differences in the command line options of passwd, usermod, and 
useradd on different Linux distributions. Verify the local files when using features like 
"disabling, suspending, or locking" on user accounts and their passwords. 


29.9. editing local files 


If you still want to manually edit the /etc/passwd or /etc/shadow, after knowing these 
commands for password management, then use vipw instead of vi(m) directly. The vipw 
tool will do proper locking of the file. 


[root@RHEL5 ~]# vipw /etc/passwd 
vipw: the password file is busy (/etc/ptmp present) 


287 


user passwords 


29.10. practice: user passwords 


1. Set the password for serena to hunter2. 


2. Also set a password for venus and then lock the venus user account with usermod. Verify 
the locking in /etc/shadow before and after you lock it. 


3. Use passwd -d to disable the serena password. Verify the serena line in /etc/shadow 
before and after disabling. 


4. What is the difference between locking a user account and disabling a user account's 
password like we just did with usermod -L and passwd -d? 


5. Try changing the password of serena to serena as serena. 
6. Make sure serena has to change her password in 10 days. 
7. Make sure every new user needs to change their password every 10 days. 


8. Take a backup as root of /etc/shadow. Use vi to copy an encrypted hunter2 hash from 
venus to serena. Can serena now log on with hunter2 as a password ? 


9. Why use vipw instead of vi ? What could be the problem when using vi or vim ? 


10. Use chsh to list all shells (only works on RHEL/CentOS/Fedora), and compare to cat / 
etc/shells. 


11. Which useradd option allows you to name a home directory ? 


12. How can you see whether the password of user serena is locked or unlocked ? Give a 
solution with grep and a solution with passwd. 
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29.11. solution: user passwords 


1. Set the password for serena to hunter2. 


root@debian7:~# passwd serena 

Enter new UNIX password: 

Retype new UNIX password: 

passwd: password updated successfully 


2. Also set a password for venus and then lock the venus user account with usermod. Verify 
the locking in /etc/shadow before and after you lock it. 


root@debian7:~# passwd venus 

Enter new UNIX password: 

Retype new UNIX password: 

passwd: password updated successfully 

root@debian7:~# grep venus /etc/shadow | cut -c1-70 
venus:$6$gswzXICWS$uSnKFVlkFKZmTPaMVSA4AvNA/KO270xNOv5LHdV9ed0gTyXr;jUeM/ 
root@debian7:~# usermod -L venus 

root@debian7:~# grep venus /etc/shadow | cut -c1-70 
venus:!$6$gswzXICWS$uSnKFVlkFKZmTPaMVSA4AvNA/KO270xNOv5LHdV9ed0gTyXr jUeM 


Note that usermod -L precedes the password hash with an exclamation mark (!). 


3. Use passwd -d to disable the serena password. Verify the serena line in /etc/shadow 


before and after disabling. 

root@debian7:~# grep serena /etc/shadow | cut -c1-70 

serena: $6SEs/omrPESF2Ypu8kpLrfKdWO0v/UIWAS5 jrYyBD2nwZ/dt.i/IypRgiPZSdB/B 
root@debian7:~# passwd -d serena 

passwd: password expiry information changed. 

root@debian7:~# grep serena /etc/shadow 


serenas: SS Ugg 9. 
root @debian7:~# 


4. What is the difference between locking a user account and disabling a user account's 
password like we just did with usermod -L and passwd -d? 


Locking will prevent the user from logging on to the system with his password by putting 
a ! in front of the password in /etc/shadow. 


Disabling with passwd will erase the password from /etc/shadow. 


5. Try changing the password of serena to serena as serena. 


log on as serena, then execute: passwd serena... it should fail! 


6. Make sure serena has to change her password in 10 days. 


chage -M 10 serena 


7. Make sure every new user needs to change their password every 10 days. 


vi /etc/login.defs (and change PASS MAX DAYS to 10) 
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8. Take a backup as root of /etc/shadow. Use vi to copy an encrypted hunter2 hash from 
venus to serena. Can serena now log on with hunter2 as a password ? 


wes. 


9. Why use vipw instead of vi ? What could be the problem when using vi or vim ? 


vipw will give a warning when someone else is already using that file (with vipw). 


10. Use chsh to list all shells (only works on RHEL/CentOS/Fedora), and compare to cat / 
etc/shells. 


chsh =i 
cat /etc/shells 


11. Which useradd option allows you to name a home directory ? 
-d 


12. How can you see whether the password of user serena is locked or unlocked ? Give a 
solution with grep and a solution with passwd. 


grep serena /etc/shadow 


passwd -S serena 
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Chapter 30. user profiles 


Logged on users have a number of preset (and customized) aliases, variables, and functions, 
but where do they come from ? The shell uses a number of startup files that are executed 


(or rather sourced) whenever the shell is invoked. What follows is an overview of startup 
scripts. 
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30.1. system profile 


Both the bash and the ksh shell will verify the existence of /etc/profile and source it if it 
exists. 


When reading this script, you will notice (both on Debian and on Red Hat Enterprise Linux) 
that it builds the PATH environment variable (among others). The script might also change 
the PS1 variable, set the HOSTNAME and execute even more scripts like /etc/inputrc 


This screenshot uses grep to show PATH manipulation in /etc/profile on Debian. 


root@debian7:~# grep PATH /etc/profile 
PATH-"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" 
PATH-"/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games" 

export PATH 

root@debian7:~# 


This screenshot uses grep to show PATH manipulation in /etc/profile on RHEL7/CentOS7. 


[root@centos7 ~]# grep PATH /etc/profile 
case ":S{PATH}:" in 
PATH=$PATH: $1 
PATH-$1:$PATH 
export PATH USER LOGNAME MAIL HOSTNAME HISTSIZE HISTCONTROL 
[root@centos7 ~]# 


The root user can use this script to set aliases, functions, and variables for every user on 
the system. 


30.2. ~/.bash_profile 


When this file exists in the home directory, then bash will source it. On Debian Linux 5/6/7 
this file does not exist by default. 


RHEL7/CentOS7 uses a small ~/.bash_profile where it checks for the existence of 
-/.bashrc and then sources it. It also adds $HOME/bin to the $PATH variable. 


[root@rhel7 ~]# cat /home/paul/.bash profile 
# .bash profile 


# Get the aliases and functions 
if [ -f ~/.bashre ]; then 
. ~/.bashre 
fi 
# User specific environment and startup programs 


PATH=$PATH: SHOME/.local/bin: SHOME/bin 


export PATH 
[root@rhel7 ~]# 
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30.3. -/.bash login 


When .bash, profile does not exist, then bash will check for -/.bash login and source it. 


Neither Debian nor Red Hat have this file by default. 


30.4. ~/.profile 


When neither -/.bash, profile and -/.bash login exist, then bash will verify the existence 
of -/.profile and execute it. This file does not exist by default on Red Hat. 


On Debian this script can execute ~/.bashre and will add SHOME/bin to the $PATH 
variable. 


root@debian7:~# tail -11 /home/paul/.profile 


if [ -n "S$SBASH VERSION" ]; then 
# include .bashrc if it exists 
if [ -f "S$HOME/.bashrc" ]; then 


"SHOME/.bashrc" 
fi 
fi 


# set PATH so it includes user's private bin if it exists 
SEEN CIS OM A te Yes 


PATH-"SHOME/bin:S$PATH" 
heal 


RHEL/CentOS does not have this file by default. 


30.5. -/.bashrc 


The ~/.bashre script is often sourced by other scripts. Let us take a look at what it does 
by default. 


Red Hat uses a very simple -/.bashrc, checking for /etc/bashre and sourcing it. It also leaves 
room for custom aliases and functions. 


[root@rhel7 ~]# cat /home/paul/.bashrc 
+ .bashrc 


4 Source global definitions 

if [ -f /etc/bashrc ]; then 
. /etc/bashrc 

BEL 


# Uncomment the following line if you don't like systemctl's auto-paging feature: 
# export SYSTEMD PAGER- 


# User specific aliases and functions 


On Debian this script is quite a bit longer and configures $PS1, some history variables and 
a number af active and inactive aliases. 


root@debian7:~# wc -1 /home/paul/.bashrc 
110 /home/paul/.bashrc 
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30.6. ^/.bash logout 


When exiting bash, it can execute -/.bash, logout. 


Debian use this opportunity to clear the console screen. 


serena@deb503:~$ cat .bash logout 
# -/.bash logout: executed by bash(1) when login shell exits. 


# when leaving the console clear the screen to increase privacy 
if [ "SSHLVL" = 1 ]; then 


[ -x /usr/bin/clear console ] && /usr/bin/clear console -q 
fi 


Red Hat Enterprise Linux 5 will simple call the /usr/bin/clear command in this script. 


[serenalrhel53 ~]$ cat .bash logout 
# -/.bash logout 


/usr/bin/clear 


Red Hat Enterprise Linux 6 and 7 create this file, but leave it empty (except for a comment). 


paul@rhel65:~$ cat .bash logout 
# -/.bash logout 
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30.7. Debian overview 


Below is a table overview of when Debian is running any of these bash startup scripts. 


Table 30.1. Debian User Environment 


[script su su - ssh gdm 
~./bashre no yes yes yes 
~/ profile no yes yes yes 
/etc/profile no yes yes yes 
| letc/bash.bashrc yes no no yes 


30.8. RHEL5 overview 


Below is a table overview of when Red Hat Enterprise Linux 5 is running any of these bash 
startup scripts. 


Table 30.2. Red Hat User Environment 

script su su - ssh gdm 
~./bashre yes yes yes yes 
~/.bash_profile no yes yes yes 
/etc/profile no yes yes yes 
letc/bashrc yes yes yes yes 
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30.9. practice: user profiles 


1. Make a list of all the profile files on your system. 

2. Read the contents of each of these, often they source extra scripts. 

3. Put a unique variable, alias and function in each of those files. 

4. Try several different ways to obtain a shell (su, su -, ssh, tmux, gnome-terminal, Ctrl- 
alt-Fl, ...) and verify which of your custom variables, aliases and function are present in 
your environment. 


5. Do you also know the order in which they are executed? 


6. When an application depends on a setting in $HOME/.profile, does it matter whether 
$HOME/.bash_profile exists or not ? 
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30.10. solution: user profiles 


1. Make a list of all the profile files on your system. 


ls a = ; Xs I /ete/pro* /etc/bash* 
2. Read the contents of each of these, often they source extra scripts. 
3. Put a unique variable, alias and function in each of those files. 


4. Try several different ways to obtain a shell (su, su -, ssh, tmux, gnome-terminal, Ctrl- 
alt-Fl, ...) and verify which of your custom variables, aliases and function are present in 
your environment. 


5. Do you also know the order in which they are executed? 
same name aliases, functions and variables will overwrite each other 


6. When an application depends on a setting in $HOME/.profile, does it matter whether 
$HOME/.bash_profile exists or not ? 


Yes it does matter. (man bash /INVOCATION) 
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Users can be listed in groups. Groups allow you to set permissions on the group level instead 
of having to set permissions for every individual user. 


Every Unix or Linux distribution will have a graphical tool to manage groups. Novice users 
are advised to use this graphical tool. More experienced users can use command line tools to 
manage users, but be careful: Some distributions do not allow the mixed use of GUI and CLI 
tools to manage groups (YaST in Novell Suse). Senior administrators can edit the relevant 
files directly with vi or vigr. 
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31.1. groupadd 


Groups can be created with the groupadd command. The example below shows the creation 
of five (empty) groups. 


root@laika:~# groupadd tennis 
root@laika:~# groupadd football 
root@laika:~# groupadd snooker 
root@laika:~# groupadd formulal 
root@laika:~# groupadd salsa 


31.2. group file 


Users can be a member of several groups. Group membership is defined by the /etc/group 
file. 


root@laika:~# tail -5 /etc/group 
tennis:x:1006: 

fooeball-sd 007: 

snooker:x:1008: 

formülal:x:1009: 

salsa:x:1010: 

root@laika:~# 


The first field is the group's name. The second field is the group's (encrypted) password (can 
be empty). The third field is the group identification or GID. The fourth field is the list of 
members, these groups have no members. 


31.3. groups 


A user can type the groups command to see a list of groups where the user belongs to. 


[harry@RHEL4b ~]$ groups 
harry sports 
[harry@RHEL4b ~]$ 
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31.4. usermod 


Group membership can be modified with the useradd or usermod command. 


root@laika:~# usermod -a -G tennis inge 
root@laika:~# usermod -a -G tennis katrien 
root@laika:~# usermod -a -G salsa katrien 
root@laika:~# usermod -a -G snooker sandra 
root@laika:~# usermod -a -G formulal annelies 
root@laika:~# tail -5 /etc/group 
tennis:x:1006:inge,katrien 

footballs LOOT: 

snooker:x:1008:sandra 
formulal:x:1009:annelies 
salsa:x:1010:katrien 

root@laika:~# 


Be careful when using usermod to add users to groups. By default, the usermod command 
will remove the user from every group of which he is a member if the group is not listed in 
the command! Using the -a (append) switch prevents this behaviour. 


31.5. groupmod 


You can change the group name with the groupmod command. 


root@laika:~# groupmod -n darts snooker 
root@laika:~# tail -5 /etc/group 
tennis:x:1006:inge, katrien 
football lsx 1007: 
formulal:x:1009:annelies 
salsa:x:1010:katrien 
darts:x:1008:sandra 


31.6. groupdel 


You can permanently remove a group with the groupdel command. 


root@laika:~# groupdel tennis 
root@laika:~# 
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31.7. gpasswd 


You can delegate control of group membership to another user with the gpasswd command. 
In the example below we delegate permissions to add and remove group members to serena 
for the sports group. Then we su to serena and add harry to the sports group. 


root@RHEL4b ~]# gpasswd -A serena sports 
root@RHEL4b ~]# su - serena 

serena@RHEL4b ~]$ id harry 

uid-516(harry) gid-520(harry) groups-520 (harry) 
serena@RHEL4b ~]$ gpasswd -a harry sports 
Adding user harry to group sports 
serena@RHEL4b ~]$ id harry 

uid=516 (harry) gid=520 (harry) groups=520 (harry) ,522 (sports) 
serena@RHEL4b ~]$ tail -1 /etc/group 
sports:x:522:serena, venus, harry 

EL4b ~]$ 


serena@RH 


Group administrators do not have to be a member of the group. They can remove themselves 
from a group, but this does not influence their ability to add or remove members. 


[serena@RHEL4b ~]$ gpasswd -d serena sports 


Removing user serena from group sports 
[serena@RHEL4b ~]$ exit 


Information about group administrators is kept in the /etc/gshadow file. 
[root@RHEL4b ~]# tail -1 /etc/gshadow 


sports: !:serena:venus, harry 
[root@RHEL4b ~]# 


To remove all group administrators from a group, use the gpasswd command to set an empty 
administrators list. 


[root@RHEL4b ~]# gpasswd -A "" sports 
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31.8. newgrp 


You can start a child shell with à new temporary primary group using the newgrp 
command. 


root@rhel65:~# mkdir prigroup 
root@rhel65:~# cd prigroup/ 
root@rhel65:~/prigroup# touch standard.txt 
root@rhel65:~/prigroup# ls -1 

total 0 
o ro e 1 ions root (0) Note LS] IET RAO) Eire leal ees 
root@rhel65:~/prigroup# echo $SHLVL 
ili 
root@rhel65:~/prigroup# newgrp tennis 
root@rhel65:~/prigroup# echo $SHLVL 
2 
root@rhel65:~/prigroup# touch newgrp.txt 
root@rhel65:~/prigroup# ls -1 


total 0 

mEWIL--r-—- d root tennis 0 Apr 13 17:49 neworp. ext 
mae O O EROS E 0 Apr 13 17:49 standard. txt 
root@rhel65:~/prigroup# exit 

exit 


root@rhel65:~/prigroup# 


31.9. vigr 


Similar to vipw, the vigr command can be used to manually edit the /etc/group file, since 
it will do proper locking of the file. Only experienced senior administrators should use vi 
or vigr to manage groups. 


302 


groups 


31.10. practice: groups 


]. Create the groups tennis, football and sports. 

2. In one command, make venus a member of tennis and sports. 

3. Rename the football group to foot. 

4. Use vi to add serena to the tennis group. 

5. Use the id command to verify that serena is a member of tennis. 


6. Make someone responsible for managing group membership of foot and sports. Test that 
it works. 
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31.11. solution: groups 


1. Create the groups tennis, football and sports. 


groupadd tennis ; groupadd football ; groupadd sports 


2. In one command, make venus a member of tennis and sports. 


usermod -a -G tennis,sports venus 


3. Rename the football group to foot. 


groupmod -n foot football 

4. Use vi to add serena to the tennis group. 

vi /etc/group 

5. Use the id command to verify that serena is a member of tennis. 


id (and after logoff logon serena should be member) 


6. Make someone responsible for managing group membership of foot and sports. Test that 
it works. 


gpasswd -A (to make manager) 


gpasswd -a (to add member) 
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Chapter 32. standard file permissions 


This chapter contains details about basic file security through file ownership and file 
permissions. 
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32.1. file ownership 


32.1.1. user owner and group owner 


The users and groups of a system can be locally managed in /etc/passwd and /etc/group, 
or they can be in a NIS, LDAP, or Samba domain. These users and groups can own files. 
Actually, every file has a user owner and a group owner, as can be seen in the following 


screenshot. 


paul@rhel65: 
total 636K 

Sew- e- == 5 
IE =e a 
LL 
IS 
paul@rhel65: 


~/owners$ ls -lh 


1 paul snooker 1.1K Apr 8 18: 
1 paul paul 626R Apr 8 Tier 
1 root tennis 185 Apr 8 18; 
JL root root @ were SBS 
~/owners$ 


47 
46 
46 
47 


data.odt 
fuse 
file2 
SCUTER EXE 


User paul owns three files; filel has paul as user owner and has the group paul as group 
owner, data.odt is group owned by the group snooker, file2 by the group tennis. 


The last file is called stuff.txt and is owned by the root user and the root group. 


32.1.2. listing user accounts 


You can use the following command to list all local user accounts. 


paul@debian7~$ cut -d: 


root 
daemon 
bin 

sys 
sync 
games 
man 

lp 

mail 
news 
uucp 
proxy 
www-data 
backup 
Site 
nm 
gnats 
nobody 
libuuid 
Debian-exim 
statd 
sshd 


ntp 
mysql 
paul 
maarten 
kevin 
yuri 
william 
yves 
kris 
hamid 
vladimir 
abiy 
david 
chahid 
stef 
joeri 
glenn 
yannick 
christof 
george 
joost 
arno 


sam 
tom 
wouter 
robrecht 
bilal 
Gamat zn 
ahmed 
dylan 
robin 
matthias 
ben 

mike 
kevin2 
kenzo 
aaron 
lorenzo 
jens 
ruben 
jelle 
stefaan 
marc 
thomas 


-f1 /etc/passwd | column 


bert 
rino 
antonio 
simon 
sven 
wouter2 
tarik 
jan 

ian 
ivan 
azeddine 
eric 
kamel 
ischa 
bart 
omer 
kurt 
steve 
constantin 
sam2 
bjorn 
ronald 


naomi 
matthias2 
bram 
fabrice 
chimene 
messagebus 
roger 
frank 
toon 
rinus 
eddy 
bram2 
keith 
jesse 
frederick 
hans 
dries 
steve2 
tomas 
johan 
tom2 
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32.1.3. chgrp 


You can change the group owner of a file using the chgrp command. 


root@rhel65:/home/paul/owners# ls -1 file2 
EW=L=>=E=>. d root tennis 185 Apr 8 18:46 file2 
root@rhel65:/home/paul/owners# chgrp snooker file2 
root@rhel65:/home/paul/owners# ls -l file2 
-rw-r--r--. 1 root snooker 185 Apr 8 18:46 file2 
root@rhel65:/home/paul/owners# 


32.1.4. chown 


The user owner of a file can be changed with chown command. 


root@laika:/home/paul# ls -l FileForPaul 

EW r- r= l root paul 0 2008=08=06 14:11 Bl GBor Bav 
root@laika:/home/paul# chown paul FileForPaul 
root@laika:/home/paul# ls -l FileForPaul 

=Ew—E——e—— l1 paul paul 0 2008=08=06 14:11 FIileForPaul 


You can also use chown to change both the user owner and the group owner. 


root@laika:/home/paul# ls -l FileForPaul 

-rw-r--r-- 1 paul paul 0 2008-08-06 14:11 FileForPaul 
root@laika:/home/paul# chown root:project42 FileForPaul 
root@laika:/home/paul# ls -l FileForPaul 

rw e E i root projectA2 0 Z 00S 00-06. I4ed TUO BIO EU 
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32.2. list of special files 


When you use Is -l, for each file you can see ten characters before the user and group owner. 
The first character tells us the type of file. Regular files get a -, directories get a d, symbolic 
links are shown with an I, pipes get a p, character devices a c, block devices a b, and sockets 
an s. 


Table 32.1. Unix special files 


first character file type 
- normal file 
d directory 
1 symbolic link 
p named pipe 
b block device 
c character device 
S socket 


Below a screenshot of a character device (the console) and a block device (the hard disk). 


paul@debian6lt~$ ls -ld /dev/console /dev/sda 
crw------- 1 root root 5, 1 Mar 15 12:45 /dev/console 
brw-rw---- 1 root disk 8, 0 Mar 15 12:45 /dev/sda 


And here you can see a directory, a regular file and a symbolic link. 


paul@debian6lt~$ ls -ld /etc /etc/hosts /etc/motd 

drwxr-xr-x 128 root root 12288 Mar 15 18:34 /etc 

-rw-r--r-- il coot COE 372 Dec 10 17:36 /etc/hosts 

lrwxrwxrwx 1 root root 13 Dee 5 10:36 /etc/motd > ad runny mot 
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32.3. permissions 
32.3.1. rwx 


The nine characters following the file type denote the permissions in three triplets. A 
permission can be r for read access, w for write access, and x for execute. You need the r 
permission to list (1s) the contents of a directory. You need the x permission to enter (cd) a 
directory. You need the w permission to create files in or remove files from a directory. 


Table 32.2. standard Unix file permissions 


permission on a file on a directory 
r (read) read file contents (cat) read directory contents (Is) 
w (write) change file contents (vi) create files in (touch) 

x (execute) execute the file enter the directory (cd) 


32.3.2. three sets of rwx 


We already know that the output of Is -1 starts with ten characters for each file. This 
screenshot shows a regular file (because the first character is a - ). 


paul@RHELv4u4:~/test$ ls -1 proc42.bash 
-rwxr-xr-- 1 paul proj 984 Feb 6 12:01 proc42.bash 


Below is a table describing the function of all ten characters. 


Table 32.3. Unix file permissions position 


position characters function 
1 - this is a regular file 
2-4 IWX permissions for the user owner 
| 5-7 T-X permissions for the group owner 
| 8-10 T-- permissions for others 


When you are the user owner of a file, then the user owner permissions apply to you. The 
rest of the permissions have no influence on your access to the file. 


When you belong to the group that is the group owner of a file, then the group owner 
permissions apply to you. The rest of the permissions have no influence on your access to 
the file. 


When you are not the user owner of a file and you do not belong to the group owner, then 
the others permissions apply to you. The rest of the permissions have no influence on your 
access to the file. 
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32.3.3. permission examples 


Some example combinations on files and directories are seen in this screenshot. The name 
of the file explains the permissions. 


paul@laika:~/perms$ ls -lh 


total 12K 
drwxr-xr-x 
—YWXIWXIWX 
Ap 
-YWXIWX-—— 
[oc S de Pr 
[ona UE 2 


Np|mnbnbNy 


paul 
paul 
paul 
paul 
paul 
paul 


paul 
paul 
paul 
paul 
paul 
paul 


paul@laika:~/perms$ 


A007 =02 07 
2007-02-07 
2007-02-07 
2007-02-07 
2007-02-07 
2007-02-07 


226 
22m 
Zoe 
pon 
22: 
Bee 


26 
Du 
eal 
al 
25 
25 


AllEnter_UserCreateDelete 
EveryoneFullControl.txt 
OnlyOwnersRead.txt 
OwnersAll_RestNothing.txt 
UserAndGroupEnter 
OnlyUserEnter 


To summarise, the first rwx triplet represents the permissions for the user owner. The 
second triplet corresponds to the group owner; it specifies permissions for all members 
of that group. The third triplet defines permissions for all other users that are not the user 
owner and are not a member of the group owner. 
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32.3.4. setting permissions (chmod) 


Permissions can be changed with chmod. The first example gives the user owner execute 
permissions. 


paul@laika:~/perms$ ls -1 permissions.txt 
At paul O 2007-02-07 27:34 permissions. tx 
paul@laika:~/perms$ chmod u+x permissions.txt 
paul@laika:~/perms$ ls -1 permissions.txt 

SEN E == == i paul) paul 0) 2007-02-07 22734 permisisions.txt 


This example removes the group owners read permission. 
paul@laika:~/perms$ chmod g-r permissions.txt 


paul@laika:~/perms$ ls -1 permissions.txt 
-rwx r l paul paul 0007020972822: S4 permise rons. txt 


This example removes the others read permission. 
paul@laika:~/perms$ chmod o-r permissions.txt 


paul@laika:~/perms$ ls -l permissions.txt 
SEWA e 1 paul paul 0 2007-02-07 22:34 permissions. txt 


This example gives all of them the write permission. 
paul@laika:~/perms$ chmod a+w permissions.txt 


paul@laika:~/perms$ ls -1 permissions.txt 
-rwx-w--w- 1 paul paul 0 2007-02-07 22:34 permissions.txt 


You don't even have to type the a. 
paul@laika:~/perms$ chmod +x permissions.txt 


paul@laika:~/perms$ ls -1 permissions.txt 
-rwx-wx-wx 1 paul paul 0 2007-02-07 22:34 permissions.txt 


You can also set explicit permissions. 
paul@laika:~/perms$ chmod u-rw permissions.txt 


paul@laika:~/perms$ ls -1 permissions.txt 
-rw--wx-wx 1 paul paul 0 2007-02-07 22:34 permissions.txt 


Feel free to make any kind of combination. 
paul@laika:~/perms$ chmod u-rw,g-rw,o-r permissions.txt 


paul@laika:~/perms$ ls -1 permissions.txt 
crw ew r dT paul paul 0 2007=02-07 22:34 permissions txt 


Even fishy combinations are accepted by chmod. 
paul@laika:~/perms$ chmod u=rwx,ug+rw, o=r permissions.txt 


paul@laika:~/perms$ ls -I permissions.txt 
-rwxrw-r-- 1 paul paul O0 2007-02-07 22:34 permissions.txt 
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32.3.5. setting octal permissions 


Most Unix administrators will use the old school octal system to talk about and set 
permissions. Look at the triplet bitwise, equating r to 4, w to 2, and x to 1. 


Table 32.4. Octal permissions 


binary octal permission 
000 0 --- 
001 1 --X 
010 2 -W- 
011 3 -WX 
100 4 T-- 
101 3 T-X 
| 110 6 rw- 
| 111 7 rwx 


This makes 777 equal to rwxrwxrwx and by the same logic, 654 mean rw-r-xr-- . The chmod 
command will accept these numbers. 


paul@laika:~/perms$ chmod 777 permissions.txt 
paul@laika:~/perms$ ls -1 permissions.txt 

-rwxrwxrwx 1 paul paul 0 2007-02-07 22:34 permissions.txt 
paul@laika:~/perms$ chmod 664 permissions.txt 
paul@laika:~/perms$ ls -l permissions.txt 

SEW ew—e—— l paul paul 0 2007-02-07 22:34 &mad5590msit5cte 
paul@laika:~/perms$ chmod 750 permissions.txt 
paul@laika:~/perms$ ls -1 permissions.txt 

=nwxr—x——— i paul) paul 0 2007-02-07 227334 permissions. txt 
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32.3.6. umask 


When creating a file or directory, a set of default permissions are applied. These default 
permissions are determined by the umask. The umask specifies permissions that you do 
not want set on by default. You can display the umask with the umask command. 


[Harry@RHEL4b -]$ umask 
0002 
[Harry@RHEL4b -]$ touch test 

[Harry@RHEL4b ~]$ ls -1 test 

-rw-rw-r-- 1 Harry Harry 0 Jul 24 06:03 test 
[Harry@RHEL4b ~]$ 


As you can also see, the file is also not executable by default. This is a general security 
feature among Unixes; newly created files are never executable by default. You have to 
explicitly do a chmod +x to make a file executable. This also means that the 1 bit in the 
umask has no meaning--a umask of 0022 is the same as 0033. 


32.3.7. mkdir -m 


When creating directories with mkdir you can use the -m option to set the mode. This 
screenshot explains. 


paul@debian5~$ mkdir -m 700 MyDir 

paul@debian5~$ mkdir -m 777 Public 

paul@debian5~S ls -dl MyDir/ Public/ 

Chap 2 paul paul 4096 2011-10-16 19:16 MyDir/ 
drwxrwxrwx 2 paul paul 4096 2011-10-16 19:16 Public/ 


32.3.8. cp -p 


To preserve permissions and time stamps from source files, use cp -p. 


paul@laika:~/perms$ cp file* cp 
paul@laika:~/perms$ cp -p file* cpp 
paul@laika:~/perms$ 11 * 


LS => 22 1 paul paul 0 2008-08-25 13:26 E1le33 
EW EX l paul paul 0 2008-08-25 13:26 file42 
Gps 

iota 

=== 1 paul paul O 2008-08-25 13:34 file33 
EW Ex == I paul paul 0 2008=08=25 13:34 flea? 
GPP: 

total 0 

epe 1 paul paul O 2008-08-25 13:26 file33 
pa l paul paul 0 2008=08=25 13:26: 9f: n] Ga? 
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32.4. practice: standard file permissions 


1. As normal user, create a directory ~/permissions. Create a file owned by yourself in there. 
2. Copy a file owned by root from /etc/ to your permissions dir, who owns this file now ? 
3. As root, create a file in the users ~/permissions directory. 

4. As normal user, look at who owns this file created by root. 

5. Change the ownership of all files in ~/permissions to yourself. 

6. Make sure you have all rights to these files, and others can only read. 

7. With chmod, is 770 the same as rwxrwx--- ? 

8. With chmod, is 664 the same as r-xr-xr-- ? 

9. With chmod, is 400 the same as r-------- ? 

10. With chmod, is 734 the same as rwxr-xr-- ? 

11a. Display the umask in octal and in symbolic form. 

11b. Set the umask to 077, but use the symbolic format to set it. Verify that this works. 


12. Create a file as root, give only read to others. Can a normal user read this file ? Test 
writing to this file with vi. 


13a. Create a file as normal user, give only read to others. Can another normal user read this 
file ? Test writing to this file with vi. 


13b. Can root read this file ? Can root write to this file with vi ? 


14. Create a directory that belongs to a group, where every member of that group can read 
and write to files, and create files. Make sure that people can only delete their own files. 
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32.5. solution: standard file permissions 


1. As normal user, create a directory ~/permissions. Create a file owned by yourself in there. 


mkdir -/permissions ; touch -/permissions/myfile.txt 


2. Copy a file owned by root from /etc/ to your permissions dir, who owns this file now ? 


cp /etc/hosts -/permissions/ 
The copy is owned by you. 


3. As root, create a file in the users ~/permissions directory. 


(become root)* touch /home/username/permissions/rootfile 


4. As normal user, look at who owns this file created by root. 


ls -1 ~/permissions 
The file created by root is owned by root. 


5. Change the ownership of all files in ~/permissions to yourself. 


chown user -/permissions/* 
You cannot become owner of the file that belongs to root. 


6. Make sure you have all rights to these files, and others can only read. 
chmod 644 (on files) 


chmod 755 (on directories) 

7. With chmod, is 770 the same as rwxrwx--- ? 
yes 

8. With chmod, is 664 the same as r-xr-xr-- ? 
No 

9. With chmod, is 400 the same as r-------- ? 
yes 

10. With chmod, is 734 the same as rwxr-xr-- ? 
no 


11a. Display the umask in octal and in symbolic form. 


umask ; umask -S 


11b. Set the umask to 077, but use the symbolic format to set it. Verify that this works. 


umask -S u-rwx,go- 
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12. Create a file as root, give only read to others. Can a normal user read this file ? Test 
writing to this file with vi. 


(become root) 

# echo hello > /home/username/root.txt 
# chmod 744 /home/username/root.txt 
(become user) 


vi Too txt 


13a. Create a file as normal user, give only read to others. Can another normal user read this 
file ? Test writing to this file with vi. 


echo hello » file ; chmod 744 file 
Yes, others can read this file 

13b. Can root read this file ? Can root write to this file with vi ? 

Yes, root can read and write to this file. Permissions do not apply to root. 


14. Create a directory that belongs to a group, where every member of that group can read 
and write to files, and create files. Make sure that people can only delete their own files. 


mkdir /home/project42 ; groupadd project42 


chgrp project42 /home/project42 ; chmod 775 /home/project42 


You can not yet do the last part of this exercise... 
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33.1. sticky bit on directory 


You can set the sticky bit on a directory to prevent users from removing files that they do 
not own as a user owner. The sticky bit is displayed at the same location as the x permission 
for others. The sticky bit is represented by a t (meaning x is also there) or a T (when there 
is no x for others). 


root@RHELv4u4:~# mkdir /project55 

root@RHELv4u4:~# ls -ld /project55 

drwxr-xr-x 2 root root 4096 Feb 7 17:38 /project55 
root@RHELv4u4:~# chmod +t /project55/ 
root@RHELv4u4:~# ls -ld /project55 

drwxr-xr-t 2 root root 4096 Feb 7 17:38 /project55 
root @RHELvV4u4: ~# 


The sticky bit can also be set with octal permissions, it is binary 1 in the first of four triplets. 


root@RHELv4u4:~# chmod 1775 /project55/ 
root@RHELv4u4:~# ls -ld /project55 

drwxrwxr-t 2 root root 4096 Feb 7 17:38 /project55 
root @RHELv4u4: ~# 


You will typically find the sticky bit on the /tmp directory. 


root@barry:~# ls -ld /tmp 
drwxrwxrwt 6 root root 4096 2009-06-04 19:02 /tmp 


33.2. setgid bit on directory 


setgid can be used on directories to make sure that all files inside the directory are owned 
by the group owner of the directory. The setgid bit is displayed at the same location as the x 
permission for group owner. The setgid bit is represented by an s (meaning x is also there) 
or a S (when there is no x for the group owner). As this example shows, even though root 
does not belong to the group proj55, the files created by root in /project55 will belong to 
proj55 since the setgid is set. 


root @RHELv4u4:~# groupadd proj55 

root@RHELv4u4:~# chown root:proj55 /project55/ 
root@RHELv4u4:~# chmod 2775 /project55/ 

root @RHELv4u4:~# touch /project55/fromroot.txt 
root@RHELv4u4:~# ls -ld /project55/ 

drwxrwsr-x 2 root proj55 4096 Feb 7 17:45 /project55/ 
root@RHELv4u4:~# ls -l /project55/ 

total 4 

=rw-r- r= Il root proJo5 0 Feb 7 17:45 fromroot. txt 
root @RHELv4u4: ~# 


You can use the find command to find all setgid directories. 


paul@laika:~$ find / -type d -perm -2000 2» /dev/null 
/var/log/mysql 

/var/log/news 

/var/local 
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33.3. setgid and setuid on regular files 


These two permissions cause an executable file to be executed with the permissions of the 
file owner instead of the executing owner. This means that if any user executes a program 
that belongs to the root user, and the setuid bit is set on that program, then the program 
runs as root. This can be dangerous, but sometimes this is good for security. 


Take the example of passwords; they are stored in /etc/shadow which is only readable by 
root. (The root user never needs permissions anyway.) 


root @RHELv4u4:~# ls -1 /etc/shadow 
-p-------- 1 root root 1260 Jan 21 07:49 /etc/shadow 


Changing your password requires an update of this file, so how can normal non-root users 
do this? Let's take a look at the permissions on the /usr/bin/passwd. 


root@RHELv4u4:~# ls -1 /usr/bin/passwd 
=E=S==X==X 1 root root 21200 Jun 17 2005 /usr/bin/passwd 


When running the passwd program, you are executing it with root credentials. 


You can use the find command to find all setuid programs. 


paul@laika:~$ find /usr/bin type f perm -04000 
/usr/bin/arping 

/usr/bin/kgrantpty 

/usr/bin/newgrp 

/usr/bin/chfn 

/usr/bin/sudo 

/usr/bin/fping6 

/usr/bin/passwd 

/usr/bin/gpasswd 


In most cases, setting the setuid bit on executables is sufficient. Setting the setgid bit will 
result in these programs to run with the credentials of their group owner. 


33.4. setuid on sudo 


The sudo binary has the setuid bit set, so any user can run it with the effective userid of root. 
paul@rhel65:~$ 1s -1 $(which sudo) 


most a l root root 23832 0ce 7 T2013 Vusr/bin/sudo 
paul@rhel65:~$ 
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33.5. practice: sticky, setuid and setgid bits 


la. Set up a directory, owned by the group sports. 

1b. Members of the sports group should be able to create files in this directory. 
Ic. All files created in this directory should be group-owned by the sports group. 
1d. Users should be able to delete only their own user-owned files. 

le. Test that this works! 


2. Verify the permissions on /usr/bin/passwd. Remove the setuid, then try changing your 
password as a normal user. Reset the permissions back and try again. 


3. If time permits (or if you are waiting for other students to finish this practice), read about 
file attributes in the man page of chattr and Isattr. Try setting the i attribute on a file and 
test that it works. 


322 


advanced file permissions 


33.6. solution: sticky, setuid and setgid bits 


la. Set up a directory, owned by the group sports. 
groupadd sports 
mkdir /home/sports 


chown root:sports /home/sports 


1b. Members of the sports group should be able to create files in this directory. 


chmod 770 /home/sports 


Ic. All files created in this directory should be group-owned by the sports group. 


chmod 2770 /home/sports 


1d. Users should be able to delete only their own user-owned files. 


chmod +t /home/sports 
le. Test that this works! 


Log in with different users (group members and others and root), create files and watch the 
permissions. Try changing and deleting files... 


2. Verify the permissions on /usr/bin/passwd. Remove the setuid, then try changing your 
password as a normal user. Reset the permissions back and try again. 


root@deb503:~# ls -1 /usr/bin/passwd 

crwst xr x 1 root root 31704 2009-11-14 15:41 /usr/bin/passwd 
root@deb503:~# chmod 755 /usr/bin/passwd 

root@deb503:~# ls -1 /usr/bin/passwd 

-rwxr-xr-x 1 root root 31704 2009-11-14 15:41 /usr/bin/passwd 


A normal user cannot change password now. 


root@deb503:~# chmod 4755 /usr/bin/passwd 
root@deb503:~# ls -1 /usr/bin/passwd 
crwsr-xr-x 1 root root 31704 2009-11-14 15:41 /usr/bin/passwd 


3. If time permits (or if you are waiting for other students to finish this practice), read about 
file attributes in the man page of chattr and Isattr. Try setting the i attribute on a file and 
test that it works. 


paul@laika:~$ sudo su - 

[sudo] password for paul: 
root@laika:~# mkdir attr 
root@laika:~# cd attr/ 
root@laika:~/attr# touch file42 
root@laika:~/attr# lsattr 

./file42 
root@laika:~/attr# chattr +i file42 
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root@laika: 
—-——-—i € P 
root@laika: 


rm: cannot 


root@laika: 
root@laika: 
root@laika: 


~/attr# lsattr 


aa = ee ./file42 
~/attr# rm -rf file42 
remove ^file42': Operation not permitted 


~/attr# chattr -i file42 
~/attr# rm -rf file42 
~/attr# 
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Standard Unix permissions might not be enough for some organisations. This chapter 
introduces access control lists or acl's to further protect files and directories. 
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34.1. acl in /etc/fstab 


File systems that support access control lists, or acls, have to be mounted with the acl 
option listed in /etc/fstab. In the example below, you can see that the root file system has 
acl support, whereas /home/data does not. 


root@laika:~# tail -4 /etc/fstab 


/dev/sdal i ext3 acl, relatime al 
/dev/sdb2 /home/data auto noacl,defaults 0 0 
pasha: /home/r /home/pasha nfs defaults 0 0 
wolf:/srv/data /home/wolf nfs defaults 0 0 


34.2. getfacl 


Reading acls can be done with /usr/bin/getfacl. This screenshot shows how to read the acl 
of file33 with getfacl. 


paul@laika:~/test$ getfacl file33 
+ files file33 

# owner: paul 

# group: paul 

user::rw- 


[SOUS E r 
mask::rwx 
[Elsa p r= 


34.3. setfacl 


Writing or changing acls can be done with /usr/bin/setfacl. These screenshots show how 
to change the acl of file33 with setfacl. 


First we add user sandra with octal permission 7 to the acl. 


paul@laika:~/testS setfacl -m u:sandra:7 file33 


Then we add the group tennis with octal permission 6 to the acl of the same file. 


paul@laika:~/testS setfacl -m g:tennis:6 file33 


The result is visible with getfacl. 


paul@laika:~/test$ getfacl file33 
# file: file33 

# owner: paul 

# group: paul 

user: :rw- 

user:sandra:rwx 

group: r= 

group:tennis:rw- 

mask::rwx 

[onis (ewe gas 
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34.4. remove an acl entry 


The -x option of the setfacl command will remove an acl entry from the targeted file. 


paulGlaika:-/test$ setfacl -m u:sandra:7 file33 
paul@laika:~/test$ getfacl file33 | grep sandra 
user: sandra: rwx 

paul@laika:~/test$S setfacl -x sandra file33 
paul@laika:~/test$ getfacl file33 | grep sandra 


Note that omitting the u or g when defining the acl for an account will default it to a user 
account. 


34.5. remove the complete acl 


The -b option of the setfacl command will remove the acl from the targeted file. 


paul@laika:~/test$ setfacl -b file33 
paul@laika:~/test$ getfacl file33 

# file: file33 

# owner: paul 

# group: paul 

user::rw- 

group: E 

other or 


34.6. the acl mask 


The acl mask defines the maximum effective permissions for any entry in the acl. This 
mask is calculated every time you execute the setfacl or chmod commands. 


You can prevent the calculation by using the --no-mask switch. 


paul@laika:~/test$ setfacl no-mask -m u:sandra:7 file33 
paul@laika:~/test$ getfacl file33 

+ files file33 

# owner: paul 

# group: paul 

user::rw- 

user:sandra:rwx #effective: rw- 

groups: rh — 

mask::rw- 

Opa 
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34.7. eiciel 


Desktop users might want to use eiciel to manage acls with a graphical tool. 


¡Basic Emblems Permissions Open With Notes Access Control List 
Access Control List 


Entry Read Write Execution E 
B paul Y Ww O 
R sandra: @ Fi Fil J 
R paul Y) O O 
2 Mak og g g 
, g Other : E: O El E 


You will need to install eiciel and nautilus-actions to have an extra tab in nautilus to 
manage acls. 


paul@laika:~$ sudo aptitude install eiciel nautilus-actions 
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An average computer using Linux has a file system with many hard links and symbolic 
links. 


To understand links in a file system, you first have to understand what an inode is. 
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35.1. inodes 


35.1.1. inode contents 


An inode is a data structure that contains metadata about a file. When the file system stores 
a new file on the hard disk, it stores not only the contents (data) of the file, but also extra 
properties like the name of the file, the creation date, its permissions, the owner of the file, 
and more. All this information (except the name of the file and the contents of the file) is 
stored in the inode of the file. 


The Is -1 command will display some of the inode contents, as seen in this screenshot. 


root8rhel53 ~# ls -ld /home/project42/ 
drwxr-xr-x 4 root pro42 4.0K Mar 27 14:29 /home/project42/ 


35.1.2. inode table 


The inode table contains all of the inodes and is created when you create the file system 
(with mkfs). You can use the df -i command to see how many inodes are used and free on 
mounted file systems. 


rootürhel53 ~# df -i 
Filesystem Inodes IUsed IFree IUse$ Mounted on 
/ dev/mapper/VolGroup00-LogVol00 


4947968 115326 4832642 a 
/dev/hdal 26104 45 26059 1% /boot 
tmpfs 64417 1 64416 1% /dev/shm 
/dev/sdal 262144 220 2599/99] 1% /home/project42 
/dev/sdb1 74400 5519 68881 8% /home/project33 
/dev/sdb5 0 0 0 - /home/sales 
/ dev/sdb6 100744 ti 100733 1% /home/research 


In the df -i screenshot above you can see the inode usage for several mounted file systems. 
You don't see numbers for /dev/sdb5 because it is a fat file system. 


35.1.3. inode number 


Each inode has a unique number (the inode number). You can see the inode numbers with 
the Is -li command. 


paul@RHELv4u4:~/test$ touch filel 
paul@RHELv4u4:~/test$ touch file2 
paul@RHELv4u4:~/test$ touch file3 
paul@RHELv4u4:~/test$ ls -li 


total 12 

817266 -rw-rw-r-- 1 paul paul 0 Feb 5 15:38 filel 
817267 —ew— > I paul paùl 0 Feb 5 15:38 tate 
817268 -rw-rw-r-- 1 paul paul 0 Feb 5 15:38 file3 


paul@RHELv4u4:~/test$ 


These three files were created one after the other and got three different inodes (the first 
column). All the information you see with this Is command resides in the inode, except for 
the filename (which is contained in the directory). 
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35.1.4. inode and file contents 


Let's put some data in one of the files. 


paul@RHELv4u4:~/test$ ls -li 


totali 

817266 mw rw- r= 1 paul paul Bel 5 15738 filet 
817270 > I paul paul 92 Feb 5 15:42 fale? 
817268 -rw-rw-r-- 1 paul paul 0 Feb 5 15:38 file3 


paul@RHELv4u4:~/test$ cat file2 

It is winter now and it is very cold. 

We do not like the cold, we prefer hot summer nights. 
paul@RHELv4u4:~/test$ 


The data that is displayed by the cat command is not in the inode, but somewhere else on 
the disk. The inode contains a pointer to that data. 


35.2. about directories 
35.2.1. a directory is a table 


A directory is a special kind of file that contains a table which maps filenames to inodes. 
Listing our current directory with Is -ali will display the contents of the directory file. 


paul@RHELv4u4:~/test$ ls -ali 


total 32 

817262 drwxrwxr-x 2 paul paul 4096 Feb 5 15:42 
900768 Gm waec 16 paul paul 4096 Feb 5 15:42 s- 
S7266 YA 1 paul paul 0 Feb 15 T5738: filel 
SLAW EW SEW E- 1 paul paul 92 Feb 5 15:42 file2 
81268 SEW EW E= 1 paul paul 0 Feb 5 15:398 files 


paul@RHELv4u4:~/test$ 
35.2.2. . and .. 


You can see five names, and the mapping to their five inodes. The dot . is a mapping to itself, 
and the dotdot .. is a mapping to the parent directory. The three other names are mappings 
to different inodes. 
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35.3. hard links 
35.3.1. creating hard links 


When we create a hard link to a file with In, an extra entry is added in the directory. A new 
file name is mapped to an existing inode. 


paul@RHELv4u4:~/test$ ln file2 hardlink to file2 
paul@RHELv4u4:~/test$ ls -li 


total 24 

817266 -rw-rw-r== 1 paul) paul 0 Heb 5 15:38 stalked 

817270 SEM SENS 2 paul paul 92 Feb 5 15:42 pale? 

817268 aw ewe l paul paul 0 Febi 5 15:38 file 

917270 =cw=nw=r-— 2 paul paul 92 Feb 5 15342 hardlink to filez 


paul@RHELv4u4:~/test$ 


Both files have the same inode, so they will always have the same permissions and the same 
owner. Both files will have the same content. Actually, both files are equal now, meaning 
you can safely remove the original file, the hardlinked file will remain. The inode contains 
a counter, counting the number of hard links to itself. When the counter drops to zero, then 
the inode is emptied. 


35.3.2. finding hard links 


You can use the find command to look for files with a certain inode. The screenshot below 
shows how to search for all filenames that point to inode 817270. Remember that an inode 
number is unique to its partition. 


paul@RHELv4u4:~/test$ find / -inum 817270 2» /dev/null 


/home/paul/test/file2 
/home/paul/test/hardlink to file2 
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35.4. symbolic links 


Symbolic links (sometimes called soft links) do not link to inodes, but create a name to 
name mapping. Symbolic links are created with In -s. As you can see below, the symbolic 


link gets an inode of its own. 


paulGRHE] 
paul@RHELv4u4:~/test$ ls -li 
total 32 

S273 EW- Ew ES ei paul 
S20) Sew EW E= 2 paul paul 
SNA Erw rw- be IE SSH paul 
S172270- EW- rw r= 2 paul paul 
817267 lrwxrwxrwx 1 paul paul 
paul@RHELv4u4:~/test$ 


Feb 
Feb 
Feb 
Feb 
Feb 


O1 O1 O! O1 O! 


17] 


107 


016 
quus 
T53 
:04 
Tes 


04 
38 
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Lv4u4:~/test$ ln -s file2 symlink to file2 


filel 

file2 

file3 

hardlink_to_file2 
symlink to file2 -> file2 


Permissions on a symbolic link have no meaning, since the permissions of the target apply. 
Hard links are limited to their own partition (because they point to an inode), symbolic links 
can link anywhere (other file systems, even networked). 


35.5. removing links 


Links can be removed with rm. 


touch data.txt 
-s data.txt sl data.txt 
data.txt hl data.txt 


paulflaika:-S 

paul@laika:~$ ln 
paul@laika:~$ ln 
paul@laika:~$ rm 
paul@laika:~$ rm 


sl data.txt 
hl data.txt 
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35.6. practice : links 


1. Create two files named winter.txt and summer.txt, put some text in them. 

2. Create a hard link to winter.txt named hlwinter.txt. 

3. Display the inode numbers of these three files, the hard links should have the same inode. 
4. Use the find command to list the two hardlinked files 

5. Everything about a file is in the inode, except two things : name them! 

6. Create a symbolic link to summer.txt called slsummer.txt. 

7. Find all files with inode number 2. What does this information tell you ? 

8. Look at the directories /etc/init.d/ /etc/rc2.d/ /etc/rc3.d/ ... do you see the links ? 

9. Look in /lib with Is -1... 


10. Use find to look in your home directory for regular files that do not(!) have one hard link. 
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35.7. solution : links 


]. Create two files named winter.txt and summer.txt, put some text in them. 


echo cold » winter.txt ; echo hot » summer.txt 

2. Create a hard link to winter.txt named hlwinter.txt. 

ln winter.txt hlwinter.txt 

3. Display the inode numbers of these three files, the hard links should have the same inode. 
ls -li winter.txt summer.txt hlwinter.txt 

4. Use the find command to list the two hardlinked files 

find . -inum xyz #replace xyz with the inode number 

5. Everything about a file is in the inode, except two things : name them! 

The name of the file is in a directory, and the contents is somewhere on the disk. 
6. Create a symbolic link to summer.txt called slsummer.txt. 

ln -s summer.txt slsummer.txt 


7. Find all files with inode number 2. What does this information tell you ? 


It tells you there is more than one inode table (one for every formatted partition + virtual 
file systems) 


8. Look at the directories /etc/init.d/ /etc/rc.d/ /etc/rc3.d/ ... do you see the links ? 
lise ece inie d 
Ls SL Veter eRe 


ls dL yextey seo el 


9. Look in /lib with Is -1... 


ls al ads 


10. Use find to look in your home directory for regular files that do not(!) have one hard link. 


aba als ES f 
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Appendix A. keyboard settings 
A.1. about keyboard layout 


Many people (like US-Americans) prefer the default US-qwerty keyboard layout. So when 
you are not from the USA and want a local keyboard layout on your system, then the best 
practice is to select this keyboard at installation time. Then the keyboard layout will always 
be correct. Also, whenever you use ssh to remotely manage a Linux system, your local 
keyboard layout will be used, independent of the server keyboard configuration. So you will 
not find much information on changing keyboard layout on the fly on linux, because not 
many people need it. Below are some tips to help you. 


A.2. X Keyboard Layout 


This is the relevant portion in /etc/X11/xorg.conf, first for Belgian azerty, then for US- 
qwerty. 


[paul@RHEL5 ~]$ grep -i xkb /etc/X11/xorg.conf 
Option "XkbModel" "pcl05" 
Option "XkbLayout" "be" 


[paul@RHEL5 ~]$ grep -i xkb /etc/X11/xorg.conf 
Option "XkbModel" "pcl05" 
Option "XkbLayout" "us" 


When in Gnome or KDE or any other graphical environment, look in the graphical menu in 
preferences, there will be a keyboard section to choose your layout. Use the graphical menu 
instead of editing xorg.conf. 


A.3. shell keyboard layout 


When in bash, take a look in the /etc/sysconfig/keyboard file. Below a sample US-qwerty 
configuration, followed by a Belgian azerty configuration. 


[paul@RHEL5 ~]$ cat /etc/sysconfig/keyboard 
KEYBOARDTYPE-"pc" 
KEYTABLE-"us" 


[paul@RHEL5 ~]$ cat /etc/sysconfig/keyboard 
KEYBOARDTYPE-"pc" 
KEYTABLE-"be-latinl" 


The keymaps themselves can be found in /usr/share/keymaps or /lib/kbd/keymaps. 


[paul@RHEL5 ~]$ ls -l /lib/kbd/keymaps/ 
total 52 
drwxr-xr-x 2 root root 4096 Apr 1 00:14 amiga 
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Appendix B. hardware 
B.1. buses 


B.1.1. about buses 


Hardware components communicate with the Central Processing Unit or cpu over a bus. 
The most common buses today are usb, pci, agp, pci-express and pcmcia aka pc-card. 
These are all Plag and Play buses. 


Older x86 computers often had isa buses, which can be configured using jumpers or dip 
switches. 


B.1.2. /proc/bus 


To list the buses recognised by the Linux kernel on your computer, look at the contents of 
the /proc/bus/ directory (screenshot from Ubuntu 7.04 and RHEL4u4 below). 


root@laika:~# ls /proc/bus/ 
input pccard pci usb 


[root@RHEL4b ~]# ls /proc/bus/ 
input pci usb 


Can you guess which of these two screenshots was taken on a laptop ? 
B.1.3. /usr/sbin/Isusb 


To list all the usb devices connected to your system, you could read the contents of /proc/ 
bus/usb/devices (if it exists) or you could use the more readable output of Isusb, which is 
executed here on a SPARC system with Ubuntu. 


root@shaka:~# lsusb 

Bus 001 Device 002: ID 0430:0100 Sun Microsystems, Inc. 3-button Mouse 
Bus 001 Device 003: ID 0430:0005 Sun Microsystems, Inc. Type 6 Keyboard 
Bus 001 Device 001: ID 04b0:0136 Nikon Corp. Coolpix 7900 (storage) 
root@shaka:~# 


B.1.4. /var/lib/usbutils/usb.ids 


The /var/lib/usbutils/usb.ids file contains a gzipped list of all known usb devices. 
paul@barry:~$ zmore /var/lib/usbutils/usb.ids | head 

== > /var/lib/usbutils/usb.ids «------ 

# List of USB ID's 


+ 
# Maintained by Vojtech Pavlik <vojtech@suse.cz> 
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If you have any new entries, send them to the maintainer. 
The latest version can be obtained from 
http://www.linux-usb.org/usb.ids 


dE Se Hb db od 


Sid: usb.ids,v 1.225 2006/07/13 04:18:02 dbrownell Exp $ 


B.1.5. /usr/sbin/Ispci 


To get a list of all pci devices connected, you could take a look at /proc/bus/pci or run lspci 
(partial output below). 


paul@laika:~$ lspci 


WO SOG. 


0 FireWire (IEEE 1394): Texas Instruments TSB43AB22/A IEEE-139... 
00:08.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-816... 
00:09.0 Multimedia controller: Philips Semiconductors SAA7133/SAA713... 
00:0a.0 Network controller: RaLink RT2500 802.11g Cardbus/mini-PCI 
00:0f.0 RAID bus controller: VIA Technologies, Inc. VIA VT6420 SATA ... 
00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A... 
00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.... 
00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.... 


B.2. interrupts 


B.2.1. about interrupts 


An interrupt request or IRQ is a request from a device to the CPU. A device raises an 
interrupt when it requires the attention of the CPU (could be because the device has data 
ready to be read by the CPU). 


Since the introduction of pci, irq's can be shared among devices. 


Interrupt 0 is always reserved for the timer, interrupt 1 for the keyboard. IRQ 2 is used as a 
channel for IRQ's 8 to 15, and thus 1s the same as IRQ 9. 


B.2.2. /proc/interrupts 


You can see a listing of interrupts on your system in /proc/interrupts. 


paul@laika:~$ cat /proc/interrupts 


CPUO CRUI 
0 1320048 555  IO-APIC-edge timer 
ries 10224 7  IO-APIC-edge 18042 
Hes 0 Q IO-APIC-edge parporto 
3: 2 1 IO-APIC-edge rtc 
iL (Ore 3062 21 IO-APIC-fasteoi acpi 
12 sat 2 IO-APIC-edge 18042 
d rs 47073 0 IO-APIC-edge idel 
18: 0 1 IO-APIC-fasteoi yenta 
198 31056 1 IO-APIC-fasteoi libata, ohcil394 
20% 19042 1 IO-APIC-fasteoi eth0 
EXE 44052 1  IO-APIC-fasteoi uhei hedkusbl- uker hed: usb ra 
225 188352 1 IO-APIC-fasteoi ra0 
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Zor) 632444 1 IO-APIC-fasteoi nvidia 
24% 1585 dl IO-APIC-fasteoi VIA82XX-MODEM, VIA8237 


B.2.3. dmesg 


You can also use dmesg to find irq's allocated at boot time. 


paul@laika:~$ dmesg | grep “arg Tras 
[ 28.930069] ata3: PATA max UDMA/133 cmd 0x1f0 ctl 0x3f6 bmdma 0x2090 irq 14 
[ 28.930071] ata4: PATA max UDMA/133 cmd 0x170 ctl 0x376 bmdma 0x2098 irq 15 


B.3. io ports 
B.3.1. about io ports 


Communication in the other direction, from CPU to device, happens through IO ports. The 
CPU writes data or control codes to the IO port of the device. But this is not only a one way 
communication, the CPU can also use a device's IO port to read status information about the 
device. Unlike interrupts, ports cannot be shared! 


B.3.2. /proc/ioports 


You can see a listing of your system's IO ports via /proc/ioports. 


[root@RHEL4b ~]# cat /proc/ioports 
0000-001f : dmal 
002020021: pici 
0040-0043 : timer0 
0050-0053 : timerl 
0060-006f : keyboard 
DONOSO 
0080-008f : dma page reg 
00a0-00al : pic2 
00c0-00df : dma2 

(IOS QOEM FpU 
Oo = vdeil 
OZES-O2ZEE o serial 


B.4. dma 
B.4.1. about dma 


A device that needs a lot of data, interrupts and ports can pose a heavy load on the cpu. With 
dma or Direct Memory Access a device can gain (temporary) access to a specific range 
of the ram memory. 


B.4.2. /proc/dma 


Looking at /proc/dma might not give you the information that you want, since it only 
contains currently assigned dma channels for isa devices. 
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root@laika:~# cat /proc/dma 
1: parportO0 
4: cascade 


pci devices that are using dma are not listed in /proc/dma, in this case dmesg can be useful. 
The screenshot below shows that during boot the parallel port received dma channel 1, and 
the Infrared port received dma channel 3. 


root@laika:~# dmesg | egrep -C 1 'dma 1|dma 3 

[ 20.576000] parport: PnPBIOS parport detected. 

[ 20.580000] parport0: PC-style at 0x378 (0x778); irg 7, dma 1... 
[ 20.764000] irda init () 


21.204000] pnp: Device 00:0b activated. 
21.204000] nsc ircc pnp probe() : From PnP, found firbase 0x2F8... 
2020/20/00] seu cc hs nate 
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Appendix C. License 


GNU Free Documentation License 
Version 1.3, 3 November 2008 
Copyright © 2000, 2001, 2002, 2007, 2008 Free Software Foundation, Inc. 


Everyone is permitted to copy and distribute verbatim copies of this 
license document, but changing it is not allowed. 


0. PREAMBLE 


The purpose of this License is to make a manual, textbook, or other 
functional and useful document "free" in the sense of freedom: to 
assure everyone the effective freedom to copy and redistribute it, 
with or without modifying it, either commercially or noncommercially. 
Secondarily, this License preserves for the author and publisher a way 
to get credit for their work, while not being considered responsible 
for modifications made by others. 


This License is a kind of "copyleft", which means that derivative 
works of the document must themselves be free in the same sense. It 
complements the GNU General Public License, which is a copyleft 
license designed for free software. 


We have designed this License in order to use it for manuals for free 
software, because free software needs free documentation: a free 
program should come with manuals providing the same freedoms that the 
Software does. But this License is not limited to software manuals; it 
can be used for any textual work, regardless of subject matter or 
whether it is published as a printed book. We recommend this License 
principally for works whose purpose is instruction or reference. 


1. APPLICABILITY AND DEFINITIONS 


This License applies to any manual or other work, in any medium, that 
contains a notice placed by the copyright holder saying it can be 
distributed under the terms of this License. Such a notice grants a 
world-wide, royalty-free license, unlimited in duration, to use that 
work under the conditions stated herein. The "Document", below, refers 
to any such manual or work. Any member of the public is a licensee, 
and is addressed as "you". You accept the license if you copy, modify 
or distribute the work in a way requiring permission under copyright 
law. 


A "Modified Version" of the Document means any work containing the 
Document or a portion of it, either copied verbatim, or with 
modifications and/or translated into another language. 


A "Secondary Section" is a named appendix or a front-matter section of 
the Document that deals exclusively with the relationship of the 
publishers or authors of the Document to the Document's overall 
subject (or to related matters) and contains nothing that could fall 
directly within that overall subject. (Thus, if the Document is in 
part a textbook of mathematics, a Secondary Section may not explain 
any mathematics.) The relationship could be a matter of historical 
connection with the subject or with related matters, or of legal, 
commercial, philosophical, ethical or political position regarding 
them. 


The "Invariant Sections" are certain Secondary Sections whose titles 
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are designated, as being those of Invariant Sections, in the notice 
that says that the Document is released under this License. If a 
section does not fit the above definition of Secondary then it is not 
allowed to be designated as Invariant. The Document may contain zero 
Invariant Sections. If the Document does not identify any Invariant 
Sections then there are none. 


The "Cover Texts" are certain short passages of text that are listed, 
as Front-Cover Texts or Back-Cover Texts, in the notice that says that 
the Document is released under this License. A Front-Cover Text may be 
at most 5 words, and a Back-Cover Text may be at most 25 words. 


A "Transparent" copy of the Document means a machine-readable copy, 
represented in a format whose specification is available to the 
general public, that is suitable for revising the document 
straightforwardly with generic text editors or (for images composed of 
pixels) generic paint programs or (for drawings) some widely available 
drawing editor, and that is suitable for input to text formatters or 
for automatic translation to a variety of formats suitable for input 
to text formatters. A copy made in an otherwise Transparent file 
format whose markup, or absence of markup, has been arranged to thwart 
or discourage subsequent modification by readers is not Transparent. 
An image format is not Transparent if used for any substantial amount 
of text. A copy that is not "Transparent" is called "Opaque". 


Examples of suitable formats for Transparent copies include plain 
ASCII without markup, Texinfo input format, LaTeX input format, SGML 
or XML using a publicly available DTD, and standard-conforming simple 
HTML, PostScript or PDF designed for human modification. Examples of 
transparent image formats include PNG, XCF and JPG. Opaque formats 
include proprietary formats that can be read and edited only by 
proprietary word processors, SGML or XML for which the DTD and/or 
processing tools are not generally available, and the 
machine-generated HTML, PostScript or PDF produced by some word 
processors for output purposes only. 


The "Title Page" means, for a printed book, the title page itself, 
plus such following pages as are needed to hold, legibly, the material 
this License requires to appear in the title page. For works in 
formats which do not have any title page as such, "Title Page" means 
the text near the most prominent appearance of the work's title, 
preceding the beginning of the body of the text. 


The "publisher" means any person or entity that distributes copies of 
the Document to the public. 


A section "Entitled XYZ" means a named subunit of the Document whose 
title either is precisely XYZ or contains XYZ in parentheses following 
text that translates XYZ in another language. (Here XYZ stands for a 
Specific section name mentioned below, such as "Acknowledgements", 
"Dedications", "Endorsements", or "History".) To "Preserve the Title" 
of such a section when you modify the Document means that it remains a 
section "Entitled XYZ" according to this definition. 


The Document may include Warranty Disclaimers next to the notice which 
states that this License applies to the Document. These Warranty 
Disclaimers are considered to be included by reference in this 
License, but only as regards disclaiming warranties: any other 
implication that these Warranty Disclaimers may have is void and has 
no effect on the meaning of this License. 


2. VERBATIM COPYING 


You may copy and distribute the Document in any medium, either 
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commercially or noncommercially, provided that this License, the 
copyright notices, and the license notice saying this License applies 
to the Document are reproduced in all copies, and that you add no 
other conditions whatsoever to those of this License. You may not use 
technical measures to obstruct or control the reading or further 
copying of the copies you make or distribute. However, you may accept 
compensation in exchange for copies. If you distribute a large enough 
number of copies you must also follow the conditions in section 3. 


You may also lend copies, under the same conditions stated above, and 
you may publicly display copies. 


3. COPYING IN QUANTITY 


If you publish printed copies (or copies in media that commonly have 
printed covers) of the Document, numbering more than 100, and the 
Document's license notice requires Cover Texts, you must enclose the 
copies in covers that carry, clearly and legibly, all these Cover 
Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on 
the back cover. Both covers must also clearly and legibly identify you 
as the publisher of these copies. The front cover must present the 
full title with all words of the title equally prominent and visible. 
You may add other material on the covers in addition. Copying with 
changes limited to the covers, as long as they preserve the title of 
the Document and satisfy these conditions, can be treated as verbatim 
copying in other respects. 


If the required texts for either cover are too voluminous to fit 
legibly, you should put the first ones listed (as many as fit 
reasonably) on the actual cover, and continue the rest onto adjacent 
pages. 


If you publish or distribute Opaque copies of the Document numbering 
more than 100, you must either include a machine-readable Transparent 
copy along with each Opaque copy, or state in or with each Opaque copy 
a computer-network location from which the general network-using 
public has access to download using public-standard network protocols 
a complete Transparent copy of the Document, free of added material. 
If you use the latter option, you must take reasonably prudent steps, 
when you begin distribution of Opaque copies in quantity, to ensure 
that this Transparent copy will remain thus accessible at the stated 
location until at least one year after the last time you distribute an 
Opaque copy (directly or through your agents or retailers) of that 
edition to the public. 


It is requested, but not required, that you contact the authors of the 
Document well before redistributing any large number of copies, to 
give them a chance to provide you with an updated version of the 
Document. 


4. MODIFICATIONS 


You may copy and distribute a Modified Version of the Document under 
the conditions of sections 2 and 3 above, provided that you release 
the Modified Version under precisely this License, with the Modified 
Version filling the role of the Document, thus licensing distribution 
and modification of the Modified Version to whoever possesses a copy 
of it. In addition, you must do these things in the Modified Version: 


* A. Use in the Title Page (and on the covers, if any) a title 
distinct from that of the Document, and from those of previous 
versions (which should, if there were any, be listed in the History 
section of the Document). You may use the same title as a previous 
version if the original publisher of that version gives permission. 
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* B. List on the Title Page, as authors, one or more persons or 
entities responsible for authorship of the modifications in the 
Modified Version, together with at least five of the principal authors 
of the Document (all of its principal authors, if it has fewer than 
five), unless they release you from this requirement. 

* C. State on the Title page the name of the publisher of the 
Modified Version, as the publisher. 

* D. Preserve all the copyright notices of the Document. 

* E. Add an appropriate copyright notice for your modifications 
adjacent to the other copyright notices. 

* F. Include, immediately after the copyright notices, a license 
notice giving the public permission to use the Modified Version under 
the terms of this License, in the form shown in the Addendum below. 

* G. Preserve in that license notice the full lists of Invariant 
Sections and required Cover Texts given in the Document's license 
notice. 

* H. Include an unaltered copy of this License. 

* I. Preserve the section Entitled "History", Preserve its Title, 
and add to it an item stating at least the title, year, new authors, 
and publisher of the Modified Version as given on the Title Page. If 
there is no section Entitled "History" in the Document, create one 
stating the title, year, authors, and publisher of the Document as 
given on its Title Page, then add an item describing the Modified 
Version as stated in the previous sentence. 

* J. Preserve the network location, if any, given in the Document 
for public access to a Transparent copy of the Document, and likewise 
the network locations given in the Document for previous versions it 
was based on. These may be placed in the "History" section. You may 
omit a network location for a work that was published at least four 
years before the Document itself, or if the original publisher of the 
version it refers to gives permission. 

* K. For any section Entitled "Acknowledgements" or "Dedications", 
Preserve the Title of the section, and preserve in the section all the 
substance and tone of each of the contributor acknowledgements and/or 
dedications given therein. 

* L. Preserve all the Invariant Sections of the Document, 
unaltered in their text and in their titles. Section numbers or the 
equivalent are not considered part of the section titles. 

* M. Delete any section Entitled "Endorsements". Such a section 
may not be included in the Modified Version. 

* N. Do not retitle any existing section to be Entitled 
"Endorsements" or to conflict in title with any Invariant Section. 

* O. Preserve any Warranty Disclaimers. 


If the Modified Version includes new front-matter sections or 
appendices that qualify as Secondary Sections and contain no material 
copied from the Document, you may at your option designate some or all 
of these sections as invariant. To do this, add their titles to the 
list of Invariant Sections in the Modified Version's license notice. 
These titles must be distinct from any other section titles. 


You may add a section Entitled "Endorsements", provided it contains 
nothing but endorsements of your Modified Version by various 
parties—-for example, statements of peer review or that the text has 
been approved by an organization as the authoritative definition of a 
standard. 


You may add a passage of up to five words as a Front-Cover Text, and a 
passage of up to 25 words as a Back-Cover Text, to the end of the list 
of Cover Texts in the Modified Version. Only one passage of 
Front-Cover Text and one of Back-Cover Text may be added by (or 
through arrangements made by) any one entity. If the Document already 
includes a cover text for the same cover, previously added by you or 
by arrangement made by the same entity you are acting on behalf of, 
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you may not add another; but you may replace the old one, on explicit 
permission from the previous publisher that added the old one. 


The author(s) and publisher(s) of the Document do not by this License 
give permission to use their names for publicity for or to assert or 
imply endorsement of any Modified Version. 


5. COMBINING DOCUMENTS 


You may combine the Document with other documents released under this 
License, under the terms defined in section 4 above for modified 
versions, provided that you include in the combination all of the 
Invariant Sections of all of the original documents, unmodified, and 
list them all as Invariant Sections of your combined work in its 
license notice, and that you preserve all their Warranty Disclaimers. 


The combined work need only contain one copy of this License, and 
multiple identical Invariant Sections may be replaced with a single 
copy. If there are multiple Invariant Sections with the same name but 
different contents, make the title of each such section unique by 
adding at the end of it, in parentheses, the name of the original 
author or publisher of that section if known, or else a unique number. 
Make the same adjustment to the section titles in the list of 
Invariant Sections in the license notice of the combined work. 


In the combination, you must combine any sections Entitled "History" 
in the various original documents, forming one section Entitled 
"History"; likewise combine any sections Entitled "Acknowledgements", 
and any sections Entitled "Dedications". You must delete all sections 
Entitled "Endorsements". 


6. COLLECTIONS OF DOCUMENTS 


You may make a collection consisting of the Document and other 
documents released under this License, and replace the individual 
copies of this License in the various documents with a single copy 
that is included in the collection, provided that you follow the rules 
of this License for verbatim copying of each of the documents in all 
other respects. 


You may extract a single document from such a collection, and 
distribute it individually under this License, provided you insert a 
copy of this License into the extracted document, and follow this 
License in all other respects regarding verbatim copying of that 
document. 


7. AGGREGATION WITH INDEPENDENT WORKS 


A compilation of the Document or its derivatives with other separate 
and independent documents or works, in or on a volume of a storage or 
distribution medium, is called an "aggregate" if the copyright 
resulting from the compilation is not used to limit the legal rights 
of the compilation's users beyond what the individual works permit. 
When the Document is included in an aggregate, this License does not 
apply to the other works in the aggregate which are not themselves 
derivative works of the Document. 


If the Cover Text requirement of section 3 is applicable to these 
copies of the Document, then if the Document is less than one half of 
the entire aggregate, the Document's Cover Texts may be placed on 
covers that bracket the Document within the aggregate, or the 
electronic equivalent of covers if the Document is in electronic form. 
Otherwise they must appear on printed covers that bracket the whole 
aggregate. 
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8. TRANSLATION 


Translation is considered a kind of modification, so you may 
distribute translations of the Document under the terms of section 4. 
Replacing Invariant Sections with translations requires special 
permission from their copyright holders, but you may include 
translations of some or all Invariant Sections in addition to the 
original versions of these Invariant Sections. You may include a 
translation of this License, and all the license notices in the 
Document, and any Warranty Disclaimers, provided that you also include 
the original English version of this License and the original versions 
of those notices and disclaimers. In case of a disagreement between 
the translation and the original version of this License or a notice 
or disclaimer, the original version will prevail. 


If a section in the Document is Entitled "Acknowledgements", 
"Dedications", or "History", the requirement (section 4) to Preserve 
its Title (section 1) will typically require changing the actual 
title. 


9. TERMINATION 


You may not copy, modify, sublicense, or distribute the Document 
except as expressly provided under this License. Any attempt otherwise 
to copy, modify, sublicense, or distribute it is void, and will 
automatically terminate your rights under this License. 


However, if you cease all violation of this License, then your license 
from a particular copyright holder is reinstated (a) provisionally, 
unless and until the copyright holder explicitly and finally 
terminates your license, and (b) permanently, if the copyright holder 
fails to notify you of the violation by some reasonable means prior to 
60 days after the cessation. 


Moreover, your license from a particular copyright holder is 
reinstated permanently if the copyright holder notifies you of the 
violation by some reasonable means, this is the first time you have 
received notice of violation of this License (for any work) from that 
copyright holder, and you cure the violation prior to 30 days after 
your receipt of the notice. 


Termination of your rights under this section does not terminate the 
licenses of parties who have received copies or rights from you under 
this License. If your rights have been terminated and not permanently 
reinstated, receipt of a copy of some or all of the same material does 
not give you any rights to use it. 


10. FUTURE REVISIONS OF THIS LICENSE 


The Free Software Foundation may publish new, revised versions of the 
GNU Free Documentation License from time to time. Such new versions 
will be similar in spirit to the present version, but may differ in 
detail to address new problems or concerns. See 
http://www.gnu.org/copyleft/. 


Each version of the License is given a distinguishing version number. 
If the Document specifies that a particular numbered version of this 
License "or any later version" applies to it, you have the option of 
following the terms and conditions either of that specified version or 
of any later version that has been published (not as a draft) by the 
Free Software Foundation. If the Document does not specify a version 
number of this License, you may choose any version ever published (not 
as a draft) by the Free Software Foundation. If the Document specifies 
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that a proxy can decide which future versions of this License can be 
used, that proxy's public statement of acceptance of a version 
permanently authorizes you to choose that version for the Document. 


11. RELICENSING 


"Massive Multiauthor Collaboration Site" (or "MMC Site") means any 
World Wide Web server that publishes copyrightable works and also 
provides prominent facilities for anybody to edit those works. A 
public wiki that anybody can edit is an example of such a server. A 
"Massive Multiauthor Collaboration" (or "MMC") contained in the site 
means any set of copyrightable works thus published on the MMC site. 


"CC-BY-SA" means the Creative Commons Attribution-Share Alike 3.0 
license published by Creative Commons Corporation, a not-for-profit 
corporation with a principal place of business in San Francisco, 
California, as well as future copyleft versions of that license 
published by that same organization. 


"Incorporate" means to publish or republish a Document, in whole or in 
part, as part of another Document. 


An MMC is "eligible for relicensing" if it is licensed under this 
License, and if all works that were first published under this License 
somewhere other than this MMC, and subsequently incorporated in whole 
or in part into the MMC, (1) had no cover texts or invariant sections, 
and (2) were thus incorporated prior to November 1, 2008. 


The operator of an MMC Site may republish an MMC contained in the site 
under CC-BY-SA on the same site at any time before August 1, 2009, 
provided the MMC is eligible for relicensing. 
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